Rough Book

random musings of just another computer nerd

Category: Software

Getting artifactory running on Ubuntu 18.04

I was trying to get the Artifactory OSS 6.3.3 running on Ubuntu 18.04 and ran into issues described in RTFACT-16909. The issue is that there are systemd changes in 18.04 that make the handling of PID files much stricter. When Artifactory starts up as a service, systemd runs /opt/jfrog/artifactory/bin/artifactoryManage.sh as root. But the script then starts up Tomcat as the artifactory user. The PID of the Tomcat process is then written to the PID file in /var/opt/jfrog/run/artifactory.pid. When control comes back to systemd, it sees that the PID file is not owned by root and refuses to deal with it. The errors look like this:

Sep 26 19:04:33 ip-172-31-41-254 artifactoryManage.sh[13784]: Max number of open files: 1024
Sep 26 19:04:33 ip-172-31-41-254 artifactoryManage.sh[13784]: Using ARTIFACTORY_HOME: /var/opt/jfrog/artifactory
Sep 26 19:04:33 ip-172-31-41-254 artifactoryManage.sh[13784]: Using ARTIFACTORY_PID: /var/opt/jfrog/run/artifactory.pid
Sep 26 19:04:33 ip-172-31-41-254 artifactoryManage.sh[13784]: Tomcat started.
Sep 26 19:05:02 ip-172-31-41-254 systemd[1]: Started Session 211 of user ubuntu.
Sep 26 19:05:12 ip-172-31-41-254 artifactoryManage.sh[13784]: Artifactory Tomcat started in normal mode
Sep 26 19:05:12 ip-172-31-41-254 systemd[1]: artifactory.service: New main PID 13844 does not belong to service, and PID file is not owned by root. Refusing.
Sep 26 19:05:12 ip-172-31-41-254 systemd[1]: artifactory.service: New main PID 13844 does not belong to service, and PID file is not owned by root. Refusing.
Sep 26 19:05:12 ip-172-31-41-254 systemd[1]: artifactory.service: Failed with result 'protocol'.
Sep 26 19:05:12 ip-172-31-41-254 systemd[1]: Failed to start Setup Systemd script for Artifactory in Tomcat Servlet Engine.
Sep 26 19:05:17 ip-172-31-41-254 systemd[1]: artifactory.service: Service hold-off time over, scheduling restart.
Sep 26 19:05:17 ip-172-31-41-254 systemd[1]: artifactory.service: Scheduled restart job, restart counter is at 201.
Sep 26 19:05:17 ip-172-31-41-254 systemd[1]: Stopped Setup Systemd script for Artifactory in Tomcat Servlet Engine.
Sep 26 19:05:17 ip-172-31-41-254 systemd[1]: Starting Setup Systemd script for Artifactory in Tomcat Servlet Engine...
Sep 26 19:05:17 ip-172-31-41-254 artifactoryManage.sh[14254]: found java executable in JAVA_HOME
Sep 26 19:05:17 ip-172-31-41-254 artifactoryManage.sh[14254]: Artifactory Tomcat already started
Sep 26 19:05:17 ip-172-31-41-254 systemd[1]: artifactory.service: Can't open PID file /var/opt/jfrog/run/artifactory.pid (yet?) after start: No such file or directory
Sep 26 19:05:17 ip-172-31-41-254 systemd[1]: artifactory.service: Failed with result 'protocol'.
Sep 26 19:05:17 ip-172-31-41-254 systemd[1]: Failed to start Setup Systemd script for Artifactory in Tomcat Servlet Engine.
Sep 26 19:05:23 ip-172-31-41-254 systemd[1]: artifactory.service: Service hold-off time over, scheduling restart.
Sep 26 19:05:23 ip-172-31-41-254 systemd[1]: artifactory.service: Scheduled restart job, restart counter is at 202.
Sep 26 19:05:23 ip-172-31-41-254 systemd[1]: Stopped Setup Systemd script for Artifactory in Tomcat Servlet Engine.

To work around this issue, you have to do the following. First, in /lib/systemd/system/artifactory.service add the following lines under the [Service] section:

User=artifactory   # change if your artifactory user is different
Group=artifactory  # change if your artifactory group is different

This will now run artifactoryManage.sh as the artifactory user. But the script assumes that it is to be run as root, and so there are some changes you will need to make. First, the script uses ulimit to change the limits on the number of open files. This will fail because the artifactory user will not have permissions to set the hard limit. You can get around that by adding the following to /etc/security/limits.conf:

artifactory soft nofile 32000  # change if your artifactory user is different
artifactory hard nofile 32000  # change if your artifactory user is different

Note: The actual numbers may be different for your system. To find out what they are, manually run artifactoryManage.sh start as the artifactory user before you make the above changes. The script should spit out lines similar to the ones above.

Finally, you will need to change some lines in /opt/jfrog/artifactory/bin/artifactoryManage.sh. In the snippet below, the commented-out line is the code as it originally appears and the modification is just below that:

#su -s "/bin/sh" ${ARTIFACTORY_USER} -c "${replicatorScript} start"
${replicatorScript} start

...

#su -s "/bin/sh" ${ARTIFACTORY_USER} -c "${replicatorScript} start"
${replicatorScript} stop

...

#su -s "/bin/sh" $ARTIFACTORY_USER -c "export JAVA_HOME='$JAVA_HOME'; $TOMCAT_HOME/bin/startup.sh"
$TOMCAT_HOME/bin/startup.sh

...

#su -s "/bin/sh" $ARTIFACTORY_USER -c "export JAVA_HOME='$JAVA_HOME'; $TOMCAT_HOME/bin/shutdown.sh"
$TOMCAT_HOME/bin/shutdown.sh

Note: I also made these exact changes to /opt/jfrog/artifactory/misc/service/artifactory for the sake of consistency, but I have not verified that it is strictly necessary.

We have to make these changes because artifactoryManage.sh is being run as the artifactory user now and so there is no need to explicitly run the other scripts as the same user. Once you make these changes, you should be able to start artifactory via systemctl start artifactory.service successfully.

Doing front-end development IS such a pain

This is so true that it’s hilarious. And sad. Any time I try to do something a little nontrivial for the front-end, it goes downhill so quickly. It’s like you are at this bizarre Home Depot with a million tools and you aren’t quite sure what they do because the most of the instruction manuals are missing pages or are just completely absent. There are very few that are complete.

You went there trying to get some nails and a hammer to hang a picture at home. But the nails only come in package that includes a lava lamp, a sledgehammer that weighs 50 lbs, and 217 blocks of assorted shapes, sizes, and colors. You get a hammer that seems to be the most popular but then you read a blog post from a carpenter ninja rockstar who has come out with a new hammer-design (and a cool name: “Hamm.ür”) that everyone is raving about and is a huge improvement over the new one and has started a new company that is already building and selling it. Home Depot also just happens to have it. You decide to get the new one; the old one wasn’t that actively supported by the manufacturer anymore anyway. In fact, it was just made by one guy in his garage and no one had seen him in a year and he rarely answered his phone or responded to emails. The new hammer does also come in a package with other stuff but at this point you really just want to hang that picture because that thing has been sitting against the wall for months.

You get home and finally start to hammer in the nail but end up burning your house down because the hammer replaced the lava lamp’s power adapter with a bare copper wire that set your curtains on fire. When you complain that using bare copper is unsafe and that they shouldn’t come with hammers or replace power adapters in a lava lamp that you didn’t even want in the first place, because seriously why the hell does anyone sell lava lamps with nails, you get told “Stfu noob! Copper is like the second-best fucking conductor and it is cheaper than silver and bare copper is so much lighter than a whole stupid adapter; seriously weren’t you just complaining about not wanting extra shit with what you buy?”

Don’t use class literals as type-tokens

Generics were added to the Java language within J2SE 5.0, and there was much rejoicing. It was finally possible to deal with containers in a type-safe manner. Prior to the availability of generics, Java developers had to do things like this:

List people = new ArrayList();
people.add(new Person("Donkey Kong"));
people.add(new Person("Guybrush Threepwood"));

Person pirate = (Person) people.get(1);

This kind of code is very fragile since it is not easy to keep track what is inside a container. If at runtime, the object you retrieve is not of the type that you’re expecting, you can get a ClassCastException. It is also remarkably easy to pollute a container by shoving objects of different types inside there, which makes it even more difficult to keep track of the types of the objects inside. Workarounds included littering code with instanceof checks, or creating a wrapper class (for example a class called PeopleList that would delegate to an internal List instance) around the container so that you could have control over the types of objects being inserted.

When generics finally arrived, people were ecstatic because now you could do things like this:

List<Person> people = new ArrayList<Person>();
people.add(new Person("Donkey Kong"));
people.add(new Person("Guybrush Threepwood"));

Person pirate = people.get(1); //It just works!

This meant no-more ugly workarounds, which means that things are awesome! Right?

Read the rest of this entry »

Heroku template for Spring 4 app with Oracle Java 8 and Tomcat 7.0.54

I’ve been playing around with Heroku at work for the past week or two. Heroku is pretty awesome if you want to get an app up and running quickly. Heroku does support Java and they have a few Java templates. Their current offering for Java uses Spring 3 and Tomcat 7.0.54 with Java 7. However, the version of Spring is somewhat older and they also use OpenJDK’s Java instead of Oracle’s Java. I wanted to try out Java 8 and also use a newer version of Spring so I upgraded the existing template to support both of those (I used a forked version of a custom buildpack for Java 8). I also had to update Heroku’s Web Runner to use Tomcat 7.0.54 (I have a pull-request waiting but I’m not sure if/when it will get approved so I have an artifact on GitHub that Maven can pull).

You can check out the template here.

How I got a medal from the Army for writing code

In 2005 my National Guard unit was deployed to Iraq as part of Operation Iraqi Freedom. My MOS (Military Occupational Specialty) in the Army was 92A, which is basically a logistics and supplies specialist. My job was to order parts for mechanics, pick them up, return old parts, manage HAZMAT, dispatch/return vehicles from missions, and handle licenses. I also did a few other things that I don’t remember right now. Anyway, at the time, the heart of this system was a tool called ULLS-G (Unit Level Logistics System – Ground). I say “at the time”, because shortly after we came back, ULLS-G was replaced by SAMS-E (Standard Army Maintenance System – Enhanced), which incidentally uses Oracle as a back-end database. Compared to SAMS-E, ULLS-G was a dinosaur. I had used it quite a bit, of course, having been in the Army for about 4 years by the time I was deployed. It was a complete pain to use it. ULLS-G was a DOS application (yes, MS-DOS) and most of the computers I used it on at the armory were only running DOS (this was circa early 2000’s so it wasn’t too uncommon to still see DOS systems around). By the time I was deployed most computers were running WinXP/2K or something like that, and so you could run ULLS-G in “MS-DOS compatibility mode”.

Read the rest of this entry »

The image that lied about itself

A few weeks ago, I ran into a puzzling issue at work. Someone was uploading an image which made it past our file-size checks, but caused an OutOfMemoryError and a heap dump when the code attempted to resize it. The information we had from the heap-dump was that it was trying to allocate memory for an image whose resolution was 18,375×11,175. What didn’t make sense is how this image was even getting through our file-size checks, because there is no way we would ever let in an image of that size.

In the code, we have a global limit for the largest file we will accept. We also have a separate limit for image uploads. If the image is over this size, but under the global limit, we will resize the image to a smaller size. The strange part was that the large image was making it past the global check, which meant that the size of the incoming data was below the global limit, but above the image-size limit. How could this be?

On a hunch, I hypothesized that perhaps the entire image wasn’t making it through. Perhaps only a part of the image was making it through with its header left intact. In an image file, there is usually a header that conveys information about the file format, the color space, and the resolution of the image! I figured that even though the data are incomplete, enough information was present in the header to enable the resizing code to make sense of it. When it tries to allocate memory for this image, based on the resolution it gets from the header, it runs out of memory!

To verify this, I manually hex-edited a file that was of proper size to have a ridiculously-large resolution. I then uploaded this file and was able to witness the behavior happening even though the file was of the proper size! So what’s the lesson here? Don’t only rely on file-size limits for images; you have to look at the resolution as well!

Maintaining a sorted array in O(1)

Yesterday, I came across an interesting question on StackOverflow. The question is as follows: assuming you have a sorted array, is it possible to increment locations by 1 (one at a time) and still ensure that the array is sorted in O(1)? Assuming you are not allowed to use any other secondary data-structures, the answer is no. This becomes evident when you have repeated values. Assume a degenerate case where you have an array of identical values; let’s say [0, 0, 0, 0, 0]. If you increment the value in location 0, you now have [1, 0, 0, 0, 0]. For this array to be sorted, the value 1 must be moved to the end of the array (index 4). This will require four comparisons, which means that this will inevitably turn into an O(n) algorithm.

How can we do better?

Read the rest of this entry »

JavaOne

I am at JavaOne! The last time I was here was in ’08, when it was run by Sun. Of course, it’s run by Oracle now. The first day has been pretty good. I attended sessions on Garbage Collection, the Nashorn JavaScript engine, writing DSLs, and about parallelization options offered by JDK 7 and 8 to leverage multicore processors. Pretty good first day!

Nashorn looks pretty interesting and I will be giving it a closer look when I come back home. They are also looking for people to help out so I am going to see if I can contribute anything useful.

Today I will be checking out a talk on Lambdas by Brian Goetz and will also be going to a talk on Big Data. Finally I also have some BOF (birds of a feather) sessions on evolutionary algorithms and writing parsers in Scala. Pretty interesting day and I’m looking forward to it!

Rendering a PDF with text-selection, using pdf.js

I have been working a project for the last few days, that deals with rendering PDF’s in-browser. Initially, I was going to parse the PDF and extract the text content, but then I ran into pdf.js, which is a library developed by Mozilla for rendering PDF’s in-browser via JavaScript. The project I am working on has a requirement that users should be able to select text within the PDF. This is possible using pdf.js. Unfortunately, the example code only shows you how to render a PDF, but not how to enable text-selection. I wasn’t able to find any API access to enable text-selection either. I finally ended up on the #pdfjs IRC channel and the friendly folks there gave me some direction. The logic for enabling text-selection was buried inside the code for Mozilla’s PDF viewer, and was heavily intertwined with the viewer code as well. I spent a few days playing around with the viewer and tracing through the code. I was stumped many times since the code was complex and I know jack about parsing PDF’s. But eventually I was able to focus on the part of the code that actually took care of enabling text-selection.

pdf.js’ approach to enabling text-selection is actually quite clever. The library overlays divs over the PDF, and these divs contain text that matches the PDF text that they are floating over. So when you select the text, you are actually selecting the text inside the overlaid divs. This was fine and dandy, but I was still stuck as far as getting this to work on my project. What I needed was a minimal example that I could adapt for my uses. After a day or two of tracing code, experimenting, debugging, and staring at the screen in frustration, I was eventually able to come up with a minimal example! To accomplish this, I extracted code that was relevant to creating the overlays out of the viewer code, into its own independent file. I also removed a lot of code that was dependent on the viewer itself. Keep in mind that this example doesn’t have functionality like text finding or matching, and that code is also heavily intertwined with the viewer code. All this example does is render a PDF with text-selection enabled. However, I think this is a good start!

If you are interested, you can check out the code on github and a working example on this fiddle.

The pertintent code is as follows (keep in mind you still require additional resources; all of that information is available on github):

window.onload = function () {
    var pdfBase64 = "..."; //base64 representing the PDF

    var scale = 1.5; //Set this to whatever you want. This is basically the "zoom" factor for the PDF.

    /**
     * Converts a base64 string into a Uint8Array
     */
    function base64ToUint8Array(base64) {
        var raw = atob(base64); //This is a native function that decodes a base64-encoded string.
        var uint8Array = new Uint8Array(new ArrayBuffer(raw.length));
        for (var i = 0; i < raw.length; i++) {
            uint8Array&#91;i&#93; = raw.charCodeAt(i);
        }

        return uint8Array;
    }

    function loadPdf(pdfData) {
        PDFJS.disableWorker = true; //Not using web workers. Not disabling results in an error. This line is
        //missing in the example code for rendering a pdf.

        var pdf = PDFJS.getDocument(pdfData);
        pdf.then(renderPdf);
    }

    function renderPdf(pdf) {
        pdf.getPage(1).then(renderPage);
    }

    function renderPage(page) {
        var viewport = page.getViewport(scale);
        var $canvas = jQuery("<canvas></canvas>");

        //Set the canvas height and width to the height and width of the viewport
        var canvas = $canvas.get(0);
        var context = canvas.getContext("2d");
        canvas.height = viewport.height;
        canvas.width = viewport.width;

        //Append the canvas to the pdf container div
        var $pdfContainer = jQuery("#pdfContainer");
        $pdfContainer.css("height", canvas.height + "px").css("width", canvas.width + "px");
        $pdfContainer.append($canvas);

        //The following few lines of code set up scaling on the context if we are on a HiDPI display
        var outputScale = getOutputScale();
        if (outputScale.scaled) {
            var cssScale = 'scale(' + (1 / outputScale.sx) + ', ' +
                (1 / outputScale.sy) + ')';
            CustomStyle.setProp('transform', canvas, cssScale);
            CustomStyle.setProp('transformOrigin', canvas, '0% 0%');

            if ($textLayerDiv.get(0)) {
                CustomStyle.setProp('transform', $textLayerDiv.get(0), cssScale);
                CustomStyle.setProp('transformOrigin', $textLayerDiv.get(0), '0% 0%');
            }
        }

        context._scaleX = outputScale.sx;
        context._scaleY = outputScale.sy;
        if (outputScale.scaled) {
            context.scale(outputScale.sx, outputScale.sy);
        }

        var canvasOffset = $canvas.offset();
        var $textLayerDiv = jQuery("<div />")
            .addClass("textLayer")
            .css("height", viewport.height + "px")
            .css("width", viewport.width + "px")
            .offset({
                top: canvasOffset.top,
                left: canvasOffset.left
            });

        $pdfContainer.append($textLayerDiv);

        page.getTextContent().then(function (textContent) {
            var textLayer = new TextLayerBuilder($textLayerDiv.get(0), 0); //The second zero is an index identifying
            //the page. It is set to page.number - 1.
            textLayer.setTextContent(textContent);

            var renderContext = {
                canvasContext: context,
                viewport: viewport,
                textLayer: textLayer
            };

            page.render(renderContext);
        });
    }

    var pdfData = base64ToUint8Array(pdfBase64);
    loadPdf(pdfData);
};
All original content on these pages is fingerprinted and certified by Digiprove
%d bloggers like this: