org.apache.commons.ssl.KeyStoreBuilder is able to convert OpenSSL style public/private keys into Java KeyStore files. It can also convert Java Keystore files into the PEM format that Apache likes.

Create a new keystore from an existing key and an existing certificate: java -cp not-yet-commons-ssl-0.3.9.jar org.apache.commons.ssl.KeyStoreBuilder PASSWORD KEY.pem CERT.pem