IcedTea-Web and OpenJDK

I posted 3 patches today

The first patch is for OpenJDK 9. It creates a hole that allows alternative plugin implementations to plug into (pardon the pun)  OpenJDK and make use of existing plugin-specific classes in OpenJDK.

The second and third patches are for IcedTea-Web. They allow IcedTea-Web to (progressively) build against OpenJDK8 and OpenJDK9.

What does this mean? Soon we might actually be able to build and run IcedTea-Web against OpenJDK. Yes, an Open Source/Free Software plugin running on top of OpenJDK (without any custom patches). Of course, a lot of this is still up in the air. The OpenJDK folks might not accept the patch, though I am hopeful they will – it’s very small and conservative.

Fun times ahead!

OpenJDK6 B29 Released!

On December 5, 2013, the OpenJDK6 b29 source bundle was published at https://java.net/projects/openjdk6/downloads/.

This is a security bugfix release that includes all the security patches since the b28 release. A complete list of changes is available too.

The test results were not ideal.

The TCK failed, with all issues again basically boiling down to limited cryptography support. There are patches available to fix this. As far as I can tell this is not a regression.

The jtreg tests did not pass cleanly either.

A special thanks to Andrew Hughes for backporting all the security fixes to OpenJDK6.

Onwards to B30!

OpenJDK6 B28 Released

On October 4, 2013, the OpenJDK6 b28 source bundle was published at https://java.net/projects/openjdk6/downloads/.

This includes all the security patches since the b27 release as well as a major version bump in hotspot to hs23. A complete list of changes is available too.

The test results were not ideal.

The TCK failed, with all issues basically boiling down to limited cryptography support. There are patches available to fix this. As far as I can tell this is not a regression.

The jtreg tests did not pass cleanly either. Most of the failures were caused by tests trying to access “javaweb.sfbay.sun.com”, which is not accessible (from outside Oracle?). Some issues were caused by tests having a @build tag but no @run tag (perhaps older jtreg versions were more lenient). Two new tests, introduced by the 2013-04-16 security update, java/beans/Introspector/7064279/Test7064279.java and java/beans/Introspector/Test5102804.java both failed.

A special thanks to Andrew Hughes for all his contributions (especially with the hotspot 23 backport) and to Pavel Tisnovsky for testing.

Onwards to B29!

Contributing Patches to OpenJDK

Someone asked me earlier today about how to contribute patches to OpenJDK. OpenJDK is probably not the easiest upstream to contribute patches to – their patch approval system is a little different from most other projects. I have tried to summarize the basic steps here. Let me know in comments if anything is not clear.

The OpenJDK Developer’s Guide covers most of this, but some of it is in too much detail and some of it is outdated. The process I describe here is what I have seen others use and what I have been using myself.

The overall process for submitting patches to OpenJDK is:

  1. Identify what group or project the fix is for
  2. Generate a webrev
  3. Send it to the right mailing list for review
  4. Make changes as requested by upstream and update webrev
  5. A sponsor will push your fix for you

As you can see, it’s not a very complicated process. Most of the delays in patch acceptance have been when the patch is non-trivial or has unexpected issues (even if those issues are known only Oracle-internal and can only be guessed by those of us outside) or someone sufficiently qualified to review the patch is not available.

One thing to note: Oracle requires signing a Contributor License Agreement before they will accept non-trivial patches.

Identify the Group or Poject

The first thing to do if you want to send a fix upstream is to identify what Group or Project it is for. For this purpose, both groups and projects are more-or-less the same (groups are more general). They consist of a collection of people who care about something. For example, some folks care about the javac compiler, some others cares about the build system and others that care about ports to different architectures. There is a list of these groups and projects available on the OpenJDK web page, on the left side.

Simply look through them and try to pick what you think is the best project or group for your fix. Some groups and projects are obvious: if you are fixing something in swing, that’s the Swing group. If you want to fix something in the ARMv8 port, that’s the aarch64 project. If you can’t find the best group, you can fall back to Core Libraries if it’s a patch for the standard java library (that is, a fix for a java.* or javax.* class) or you can fall back to Hotspot if it is a fix for hotspot.

Make a note of the mailing list(s) and repositories used by the group.

Don’t worry if you can’t find the ideal group, project or repository; someone will point you in the right direction in later on, but it may mean (slightly) more work later.

Generate a webrev

Since you are trying to submit a patch upstream, I will assume you already have a fix for it. The first thing you need to do is to get the latest OpenJDK code using the most appropriate repository. And then make your fix. Once you have built and tested your fix, you can generate a patch for review. OpenJDK uses a custom tool called webrev to generate easier-to-reivew patches.

Assuming you are making a fix to hotspot, use it like so:

cd jdk8/hotspot
ksh ../make/scripts/webrev.ksh

This program will look for changes in the current repository and generate a bunch of files. You should see a new
webrev.zip file and a webrev directory. The webrev directory should contain an index.html file that should look roughly like this existing one

Upload the webrev directory somewhere publicly where anyone who wants to review the patch can see it. A public dropbox folder works just fine, as done any personal webspace that you may have. Regular contributors to OpenJDK generally use cr.openjdk.java.net.

Send mail with patch

Now you need to send an email describing the patch and what it’s fixing to the right OpenJDK mailing list. remember to subscribe to the mailing list first, though.

Use a descriptive subject in the email, prefixing it with “RFR” (or Request for Review). Link to the public webrev in the email.

Make changes and update webrev

Hopefully, someone will review the patch quickly (depending on the fix, you may get a response within the hour or it may take a few days).

The reviewer may suggest improvements or additional fixes. They may also request additional cleanup or testing as well.

A Sponsor will push the fix

Once upstream is happy with the patch, a sponsor will ask you for a changeset that they can push or perhaps they will generate one for you. The details of producing a changeset are available in the developer’s guide. Since you don’t have push access, a sponsor will push the fix on your behalf to the appropriate repository. It will probably take a few weeks until that change makes it from group repository to integration repository to the master repository (jdk8/jdk8).

And that’s it! So don’t be afraid to propose patches for OpenJDK :)

Complete details about all this process is available, of course, in the Developer’s Guide.

SharedSecrets in OpenJDK

I have always been impressed by the effort OpenJDK folks go to maintain API compatibility. I recently came across a nifty little trick that is used in the OpenJDK codebase to help achieve this goal. Behold the SharedSecrets class! The name of the class is a little misleading since it doesn’t have anything to do with the cryptographic concepts of shared secrets. The javadocs do a (slightly) better job describing its purpose:

/** A repository of "shared secrets", which are a mechanism for
    calling implementation-private methods in another package without
    using reflection. A package-private class implements a public
    interface and provides the ability to call package-private methods
    within that package; the object implementing that interface is
    provided through a third package to which access is restricted.
    This framework avoids the primary disadvantage of using reflection
    for this purpose, namely the loss of compile-time checking. */

In other words, it is used to make private methods public, without using reflection! AND it works under a security manager! At first, I thought this was too good to be true but it turns out that the concept behind it is pretty simple. Here’s how it works. Suppose we have a class with a fixed public api and we wish to add a new method to it that we would like to be able to use internally within the project but not provide as an api:

public class FixedPublicApi {
  private static doFoo() { /* do something */ }
}

We could choose to add the doFoo method to another class (in fact, it will primarily be accessed through another class) and make it public, but let’s assume that the only way to avoid duplicating lots of code is to keep doFoo in FixedPublicApi. First we create an interface that exposes the method we want to be invoked:

public interface FooHandler {
  public void doFoo();
}

Then we modify the SharedSecrets class (or it’s equivalent in our codebase), to set the implementation for FooHandler

public class SharedSecrets {
  private static FooHandler fooHandler;
  public static void setFoo(FooHandler handler) {
    fooHandler = handler;
  }
  public static void foo() {
    fooHandler.doFoo();
  }
}

Finally we modify the FixedPublicApi class to install a handler:

public class FixedPublicApi {
  static {
     SharedSecrets.setFoo(new FooHandler() {
       public void doFoo() {
         FixedPublicApi.doFoo();
       }
     }
  }

  private static doFoo() { /* do something */ }
}

And that’s it! Now we can use SharedSecrets to call the private method:

  SharedSecrets.doFoo();

Hats off to whoever came up with this great technique.

OpenJDK and Awesome

I have been playing around with awesome and ran into some issues with Java programs (apparently others users are running into those problems too). I hacked together a patch against OpenJDK7 that seems to solve all my issues:

diff --git a/src/solaris/classes/sun/awt/X11/XWM.java b/src/solaris/classes/sun/awt/X11/XWM.java
--- a/src/solaris/classes/sun/awt/X11/XWM.java
+++ b/src/solaris/classes/sun/awt/X11/XWM.java
@@ -102,7 +102,8 @@
         METACITY_WM = 11,
         COMPIZ_WM = 12,
         LG3D_WM = 13,
-        CWM_WM = 14;
+        CWM_WM = 14,
+        AWESOME_WM = 15;
     public String toString() {
         switch  (WMID) {
           case NO_WM:
@@ -131,6 +132,8 @@
               return "LookingGlass";
           case CWM_WM:
               return "CWM";
+          case AWESOME_WM:
+              return "awesomewm";
           case UNDETERMINED_WM:
           default:
               return "Undetermined WM";
@@ -573,8 +576,12 @@
 //                            getIntProperty(XToolkit.getDefaultRootWindow(), XAtom.XA_CARDINAL)) == 0);
     }
 
+    static boolean isAwesomeWM() {
+        return isNetWMName("awesome");
+    }
+
     static boolean isNonReparentingWM() {
-        return (XWM.getWMID() == XWM.COMPIZ_WM || XWM.getWMID() == XWM.LG3D_WM || XWM.getWMID() == XWM.CWM_WM);
+        return (XWM.getWMID() == XWM.COMPIZ_WM || XWM.getWMID() == XWM.LG3D_WM || XWM.getWMID() == XWM.CWM_WM || XWM.getWMID() == XWM.AWESOME_WM);
     }
 
     /*
@@ -754,6 +761,8 @@
                 awt_wmgr = CWM_WM;
             } else if (doIsIceWM && isIceWM()) {
                 awt_wmgr = XWM.ICE_WM;
+            } else if (isAwesomeWM()) {
+                awt_wmgr = XWM.AWESOME_WM;
             }
             /*
              * We don't check for legacy WM when we already know that WM


Given the effort it’s taking to push a fix for mutter (the default gnome3 window manager), I wouldnt hold my breath waiting for this code to make it upstream.

IcedTea6 1.11.1 Released!

A new release of IcedTea6 1.11 branch is now available: 1.11.1

The update contains the following security fixes:

- S7082299, CVE-2011-3571: Fix in AtomicReferenceArray
– S7088367, CVE-2011-3563: Fix issues in java sound
– S7110683, CVE-2012-0502: Issues with some KeyboardFocusManager method
– S7110687, CVE-2012-0503: Issues with TimeZone class
– S7110700, CVE-2012-0505: Enhance exception throwing mechanism in
ObjectStreamClass
– S7110704, CVE-2012-0506: Issues with some method in corba
– S7112642, CVE-2012-0497: Incorrect checking for graphics rendering
object
– S7118283, CVE-2012-0501: Better input parameter checking in zip file
processing
– S7126960, CVE-2011-5035: (httpserver) Add property to limit number
of request headers to the HTTP Server

This release also contains the following additional fix:
– PR865: Patching fails with patches/ecj/jaxws-getdtdtype.patch

The tarball can be downloaded from:

http://icedtea.classpath.org/download/source/icedtea6-1.11.1.tar.gz

SHA256SUM:
bafb0e21e1edf5ee22871b13dbc0a8a0d3efd894551fb91d5f59783069b6912c

A signature (produced using my public key) is available at:

http://icedtea.classpath.org/download/source/icedtea6-1.11.1.tar.gz.sig

The following people helped with this release:

* Andrew Haley
* Andrew John Hughes
* Chris Phillips
* Danesh Dadachanji
* Deepak Bhole
* Jiri Vanek
* Omair Majid
* Roman Kennke

A huge thanks to everyone who helped test this release and reported bugs!

To get started:
$ tar xf icedtea6-1.11.1.tar.gz
$ cd icedtea6-1.11.1

Full build requirements and instructions are in INSTALL:
$ ./configure [–enable-zero –enable-pulse-java –enable-systemtap …]
$ make