Superpatterns Pat Patterson on the Cloud, Identity and Single Malt Scotch

29Jan/180

Salesforce Mutual Authentication – Part 2: Web Service Connector (WSC)

CodeIn my last blog entry I explained how to enable, configure and test Salesforce's Mutual Authentication feature. This time, I'll share my experience getting Mutual Authentication working with the Java client SDK for Salesforce's SOAP and Bulk APIs: Web Service Connector, aka WSC.

StreamSets Data Collector's Salesforce integration accesses the SOAP and Bulk APIs via WSC, so, when I was implementing Mutual Authentication in SDC, I examined WSC to see where I could configure the client key and certificate chain. Although there is no mention of SSLContext or SSLSocketFactory in the WSC code, it is possible to set a custom TransportFactory on the WSC ConnectorConfig object. The TransportFactory is used to create a Transport, which in turn is responsible for making the HTTPS connection to Salesforce.

To enable Mutual Authentication I would need to create an SSLContext with the client key and certificate chain. This is straightforward enough:

// Make a KeyStore from the PKCS-12 file
KeyStore ks = KeyStore.getInstance("PKCS12");
try (FileInputStream fis = new FileInputStream(KEYSTORE_PATH)) {
  ks.load(fis, KEYSTORE_PASSWORD.toCharArray());
}

// Make a KeyManagerFactory from the KeyStore
KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
kmf.init(ks, KEYSTORE_PASSWORD.toCharArray());

// Now make an SSL Context with our Key Manager and the default Trust Manager
SSLContext sslContext = SSLContext.getInstance("TLS");
sslContext.init(kmf.getKeyManagers(), null, null);

Given the SSLContext, we can create an SSLSocketFactory and set it on the HttpsURLConnection. Here's the code we'd use if we were simply using the java.net classes directly:

URL url = new URL(someURL);
HttpURLConnection conn = (HttpURLConnection)url.openConnection();
// Check that we did get an HttpsURLConnection before casting to it
if (conn instanceof HttpsURLConnection) {
  ((HttpsURLConnection)conn).setSSLSocketFactory(
      sslContext.getSocketFactory()
  );
}

Mutual Authentication and the Salesforce SOAP API

The default Transport implementation, JdkHttpTransport, looked like a good place to start. My first thought was to extend JdkHttpTransport, overriding the relevant methods. Unfortunately, JdkHttpTransport's createConnection method, which calls url.openConnection(), is static, so it's impossible to override. The connectRaw() method also looked like a promising route, since it calls createConnection(), performs some setup on the HttpURLConnection, and then gets the OutputStream, but it's private, and once the OutputStream has been created, it's too late to set the SSLSocketFactory.

In my searching for an answer, I came across this comment from Salesforce Software Engineer Steven Lawrance in a Salesforce Trailblazer Community answer.

You'll generally need to set the TransportFactory in the ConnectorConfig object that you use to create the PartnerConnection (or EnterpriseConnection, etc), though another option is to set the Transport.

It's possible to create a Transport implementation that is based off of the com.sforce.ws.transport.JdkHttpTransport class while having the JdkHttpTransport create the connection with its static createConnection method. Your Transport implementation can then set up the SSLSocketFactory (casting the connection to HttpsURLConnection is required to do that), and your SSLSocketFactory can be created from creating an SSLContext that is initialized to include your client certificate.

I followed Steven's advice and created ClientSSLTransport, a clone of JdkHttpTransport, and ClientSSLTransportFactory, its factory class. To minimize the amount of copied code, I changed the implementation of connectRaw() to call JdkHttpTransport.createConnection() and then set the SSLSocketFactory:

private OutputStream connectRaw(String uri, HashMap<String, String> httpHeaders, boolean enableCompression)
throws IOException {
  url = new URL(uri);

  connection = JdkHttpTransport.createConnection(config, url, 
      httpHeaders, enableCompression);
  if (connection instanceof HttpsURLConnection) {
    ((HttpsURLConnection)connection).setSSLSocketFactory(
        sslContext.getSocketFactory()
    );
  }
  connection.setRequestMethod("POST");
  connection.setDoInput(true);
  connection.setDoOutput(true);
  if (config.useChunkedPost()) {
    connection.setChunkedStreamingMode(4096);
  }

  return connection.getOutputStream();
}

With this in place, I wrote a simple test application to call an API with Mutual Authentication. As I mentioned in the previous blog post, the Salesforce login service does not support Mutual Authentication, so the inital code to authenticate is just the same as the default case:

// Login as normal to get instance URL and session token
ConnectorConfig config = new ConnectorConfig();
config.setAuthEndpoint("https://login.salesforce.com/services/Soap/u/39.0");
config.setSslContext(sc);
config.setUsername(USERNAME);
config.setPassword(PASSWORD);

connection = Connector.newConnection(config);

// display some current settings
System.out.println("Auth EndPoint: "+config.getAuthEndpoint());
System.out.println("Service EndPoint: "+config.getServiceEndpoint());

Running this bit of code revealed that, not only does the login service not support Mutual Authentication, it returns the default service endpoint:

Auth EndPoint: https://login.salesforce.com/services/Soap/u/39.0
Service EndPoint: https://na30.salesforce.com/services/Soap/u/39.0/00D36000000psQd

Before we can call an API, then, we have to override the service endpoint, changing the port from the default 443 to 8443, as well as setting the TransportFactory:

String serviceEndpoint = config.getServiceEndpoint();
// Override service endpoint port to 8443
config.setServiceEndpoint(changePort(serviceEndpoint, 8443));

// Set custom transport factory
config.setTransportFactory(new ClientSSLTransportFactory(sslContext));

...

private static String changePort(String url, int port) throws URISyntaxException {
  URI uri = new URI(url);
  return new URI(
      uri.getScheme(), uri.getUserInfo(), uri.getHost(),
      port, uri.getPath(), uri.getQuery(), uri.getFragment()).toString();
}

With this in place, I could call a SOAP API in the normal way:

System.out.println("Querying for the 5 newest Contacts...");

// query for the 5 newest contacts
QueryResult queryResults = connection.query("SELECT Id, FirstName, LastName, Account.Name " +
    "FROM Contact WHERE AccountId != NULL ORDER BY CreatedDate DESC LIMIT 5");
if (queryResults.getSize() > 0) {
  for (SObject s: queryResults.getRecords()) {
    System.out.println("Id: " + s.getId() + " " + s.getField("FirstName") + " " +
        s.getField("LastName") + " - " + s.getChild("Account").getField("Name"));
  }
}

With output:

Querying for the 5 newest Contacts...
Id: 00336000009BusFAAS Rose Gonzalez - Edge Communications
Id: 00336000009BusGAAS Sean Forbes - Edge Communications
Id: 00336000009BusHAAS Jack Rogers - Burlington Textiles Corp of America
Id: 00336000009BusIAAS Pat Stumuller - Pyramid Construction Inc.
Id: 00336000009BusJAAS Andy Young - Dickenson plc

Success!

Mutual Authentication and the Salesforce Bulk API

Now, what about the Bulk API? Running a test app resulted in an error when I tried to create a Bulk API Job. Tracing through the WSC code revealed that when ConnectorConfig.createTransport() creates a Transport with a custom TransportFactory, it does not set the ConnectorConfig on the Transport:

public Transport createTransport() throws ConnectionException {
  if(transportFactory != null) {
    return transportFactory.createTransport();
  }

  try {
    Transport t = (Transport)getTransport().newInstance();
    t.setConfig(this);
    return t;
  } catch (InstantiationException e) {
    throw new ConnectionException("Failed to create new Transport " + getTransport());
  } catch (IllegalAccessException e) {
    throw new ConnectionException("Failed to create new Transport " + getTransport());
  }
}

ConnectorConfig.createTransport() is only used when the WSC Bulk API client is POSTing to the Bulk API, since the POST method is hardcoded into JdkHttpTransport.connectRaw() (all SOAP requests use HTTP POST). When the client wants to do a GET, it uses BulkConnection.doHttpGet(), which does not use ConnectorConfig.createTransport(), instead calling config.createConnection():

private InputStream doHttpGet(URL url) throws IOException, AsyncApiException {
  HttpURLConnection connection = config.createConnection(url, null);
  connection.setRequestProperty(SESSION_ID, config.getSessionId());
  ...

The problem here is that config.createConnection() ultimately just calls url.openConnection() directly, bypassing any custom Transport:

public HttpURLConnection createConnection(URL url,
HashMap<String, String> httpHeaders, boolean enableCompression) throws IOException {

  if (isTraceMessage()) {
    getTraceStream().println( "WSC: Creating a new connection to " + url + " Proxy = " +
        getProxy() + " username " + getProxyUsername());
  }

  HttpURLConnection connection = (HttpURLConnection) url.openConnection(getProxy());
  connection.addRequestProperty("User-Agent", VersionInfo.info());
  ...

Luckily, config.createConnection() is public, so my solution to these problems was to subclass ConnectorConfig as MutualAuthConnectorConfig, providing an SSLContext in its constructor, and overriding createConnection():

public class MutualAuthConnectorConfig extends ConnectorConfig {
  private final SSLContext sc;

  public MutualAuthConnectorConfig(SSLContext sc) {
    this.sc = sc;
  }

  @Override
  public HttpURLConnection createConnection(URL url, HashMap<String, String> httpHeaders, 
      boolean enableCompression) throws IOException {
    HttpURLConnection connection = super.createConnection(url, httpHeaders, enableCompression);
    if (connection instanceof HttpsURLConnection) {
      ((HttpsURLConnection)connection).setSSLSocketFactory(sc.getSocketFactory());
    }
    return connection;
  }
}

If you look at ClientSSLTransport and ClientSSLTransportFactory, you'll notice that the factory has a two-argument constructor that allows us to pass the ConnectorConfig. This ensures that the Transport can get the configuration it needs, despite the fact that ConnectorConfig.createTransport() neglects to set the config.

Now, when creating a BulkConnection from a Partner API ConnectorConfig, I use my subclassed ConnectorConfig class AND set the TransportFactory on it, so that the SSLSocketFactory is set for both GET and POST:

  ConnectorConfig bulkConfig = new MutualAuthConnectorConfig(sslContext);
  bulkConfig.setTransportFactory(new ClientSSLTransportFactory(sslContext, bulkConfig));
  bulkConfig.setSessionId(partnerConfig.getSessionId()); 

  // The endpoint for the Bulk API service is the same as for the normal 
  // SOAP uri until the /Soap/ part. From here it's '/async/versionNumber' 
  String soapEndpoint = partnerConfig.getServiceEndpoint(); 
  String restEndpoint = soapEndpoint.substring(0, soapEndpoint.indexOf("Soap/")) 
      + "async/" + conf.apiVersion; 

  // Remember to swap the port for Mutual Authentication! 
  bulkConfig.setRestEndpoint(changePort(restEndpoint, 8443));

Running my simple sample app showed that I was able to successfully retrieve data via the Bulk API:

Querying for the 5 newest Contacts via the Bulk API...
Created job: 7503600000KbCyMAAV
Batch state is: Queued
Sleeping for a second...
Sleeping for a second...
Sleeping for a second...
Batch state is: Completed
Result header:[Id, FirstName, LastName, Account.Name]
Id: 00336000009BusFAAS Rose Gonzalez - Edge Communications
Id: 00336000009BusGAAS Sean Forbes - Edge Communications
Id: 00336000009BusHAAS Jack Rogers - Burlington Textiles Corp of America
Id: 00336000009BusIAAS Pat Stumuller - Pyramid Construction Inc.
Id: 00336000009BusJAAS Andy Young - Dickenson plc

You can grab my sample app and all of the above mentioned files here.

Proposed WSC Changes

With the above changes I was able to call both the SOAP and Bulk APIs and include the WSC JAR files unchanged. I filed issue #213 on WSC, and then fixed the problems in the WSC directly (pull request) by adding an SSLContext member variable and its getter/setter to ConnectorConfig and having JdkHttpTransport.connectRaw() and BulkConnection.doHttpGet() set the SSLSocketFactory on the HttpsURLConnection immediately after it's created. I'll update this blog entry if and when my pull request is accepted.

Conclusion

The first blog entry in this series explained how to enable, configure and test Salesforce Mutual Authentication. This time, I showed how to work around the shortcomings in the Salesforce Web Service Connector (WSC) to allow it to work with Mutual Authentication.

In part 3, the final installment in this series, I show you how to use Mutual Authentication with common HTTP clients to access Salesforce API endpoints directly.

25Jan/182

Salesforce Mutual Authentication – Part 1: the Basics

Mutual Authentication was introduced by Salesforce in the Winter '14 release. As the Salesforce Winter '14 release notes explain,  mutually authenticated transport layer security (TLS) allows secure server-to-server connections initiated by a client using client certificate authentication, and means that both the client and the server authenticate and verify that they are who they say they are. In this blog post, I'll show you how to enable Mutual Authentication and perform some basic tests using the curl command line tool. In a future blog post, I'll show you how to implement Mutual Authentication in your Java apps.

In the default case, without Mutual Authentication, when an API client connects to Salesforce via TLS, the client authenticates the server via its TLS certificate, but the TLS connection itself gives the server no information on the client's identity. After the TLS session is established, the client sends a login request containing its credentials over the secure channel, the Salesforce login service responding with a session ID. The client then sends this session ID with each API request.

Mutual Authentication provides an additional layer of security. Each time you connect to a Salesforce API, the server checks that the client's certificate is valid for the client's org, as well as checking the validity of the session ID. Note that Mutual Authentication is intended for API use and not for user interface (web browser) use.

Before you can use Mutual Authentication, you need to obtain a client certificate. This certificate must be issued by a certificate authority with its root certificate in the Salesforce Outbound Messaging SSL CA Certificates list; Mutual Authentication will not work with a self-signed client certificate. More information is available in the Salesforce document, Set Up a Mutual Authentication Certificate. I bought an SSL certificate from GoDaddy - you can almost certainly find a cheaper alternative if you spend some time looking.

Enabling Mutual Authentication in Salesforce

Mutual Authentication is not enabled by default. You must open a support case with Salesforce to enable it. When it is enabled, you will see a Mutual Authentication Certificates section at Setup | Administer | Security Controls | Certificate and Key Management.

Mutual Authentication Configuration

You must upload a PEM-encoded client certificate to this list. Note that you need only upload the client certificate itself; do not upload a certificate chain.

You will also need to create a user profile with the Enforce SSL/TLS Mutual Authentication user permission enabled. Clone an existing Salesforce profile and enable Enforce SSL/TLS Mutual Authentication. Check that the profile has the Salesforce object permissions that your application will need to access data. Assign the new profile to the user which your app will use to access Salesforce.

Testing Mutual Authentication with curl

This was a stumbling block for me for some time. First, despite what the Salesforce documentation (Configure Your API Client to Use Mutual Authentication) says, the Salesforce login service does not support Mutual Authentication. You cannot connect to login.salesforce.com on port 8443 as described in the docs.

You can, however, send a normal authentication request for a user with Enforce SSL/TLS Mutual Authentication enabled to the default TLS port, 443. The login service responds with a session ID as for any other login request. Mutual Authentication is enforced when you use the session ID with an API endpoint.

Let's try this out. Here's a SOAP login request - add a username/password and save it to login.xml:

<?xml version="1.0" encoding="utf-8" ?>
<env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xmlns:env="http://schemas.xmlsoap.org/soap/envelope/">
  <env:Body>
    <n1:login xmlns:n1="urn:partner.soap.sforce.com">
      <n1:username>user@example.com</n1:username>
      <n1:password>p455w0rd</n1:password>
    </n1:login>
  </env:Body>
</env:Envelope>

Now you can send it to the login service with curl:

$ curl -s -k https://login.salesforce.com/services/Soap/u/41.0 \
    -H "Content-Type: text/xml; charset=UTF-8" \
    -H "SOAPAction: login" \
    -d @login.xml | xmllint --format -
<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns="urn:partner.soap.sforce.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soapenv:Body>
    <loginResponse>
      <result>
        <metadataServerUrl>https://na30.salesforce.com/services/Soap/m/41.0/00D36000000bLGT</metadataServerUrl>
        <passwordExpired>false</passwordExpired>
        <sandbox>false</sandbox>
        <serverUrl>https://na30.salesforce.com/services/Soap/u/41.0/00D36000000bLGT</serverUrl>
        <sessionId>00D36000000bLGT!AQQAQMlp30Zpiy6_gSeP1cmQG.0dHlfMcUDg96d8BSRpSb9BwksAABdKsde14ahtDGzKzRXAMroiomST8.UWcg.hp5XXDi4O</sessionId>
        <userId>00536000006Z51jAAC</userId>
        <userInfo>
          ...lots of user data...
        </userInfo>
      </result>
    </loginResponse>
  </soapenv:Body>
</soapenv:Envelope>

We need to create a PEM file for curl with the signing key, client certificate, and all the certificates in its chain except the root. This file looks something like this:

-----BEGIN RSA PRIVATE KEY-----
...base 64 encoded private key data...
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
...base64 encoded client certificate data...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...base64 encoded CA issuing cert...
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
...another base64 encoded CA issuing cert...
-----END CERTIFICATE-----

We'll call the getUserInfo API. Here's the SOAP request - add the session ID returned from login and save it as getuserinfo.xml:

<?xml version="1.0" encoding="utf-8"?> 
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
 xmlns:urn="urn:partner.soap.sforce.com">
  <soapenv:Header>
    <urn:SessionHeader>
      <urn:sessionId>INSERT_YOUR_SESSION_ID_HERE</urn:sessionId>
    </urn:SessionHeader>
  </soapenv:Header>
  <soapenv:Body>
    <urn:getUserInfo />
  </soapenv:Body>
</soapenv:Envelope>

Now we're ready to make a mutually authenticated call to a Salesforce API! You'll need to specify the correct instance, as returned in the login response, in the URL. Note the port number is 8443:

$ curl -s -k https://na30.salesforce.com:8443/services/Soap/u/41.0 \
    -H "Content-Type: text/xml; charset=UTF-8" \
    -H "SOAPAction: example" \
    -d @getuserinfo.xml \
    -E fullcert.pem | xmllint --format -
<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns="urn:partner.soap.sforce.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soapenv:Header>
    <LimitInfoHeader>
      <limitInfo>
        <current>6</current>
        <limit>15000</limit>
        <type>API REQUESTS</type>
      </limitInfo>
    </LimitInfoHeader>
  </soapenv:Header>
  <soapenv:Body>
    <getUserInfoResponse>
      <result>
        ...all the user data...
      </result>
    </getUserInfoResponse>
  </soapenv:Body>
</soapenv:Envelope>

Now let's look at a couple of failure modes. What happens when we call the 8443 port, but don't pass a client certificate?

$ curl -s -k https://na30.salesforce.com:8443/services/Soap/u/41.0 \
    -H "Content-Type: text/xml; charset=UTF-8" \
    -H "SOAPAction: example" \
    -d @getuserinfo.xml
<html><head><title>Certificate Error</title></head><body bgcolor=#ffffff text=#3198d8><center><img src="http://www.sfdcstatic.com/common/assets/img/logo-company.png"><p><h3>Client certificate error:<i>No client certificate provided.</i></h3></center></body></html>

Note the HTML response, rather than XML!

What about calling the regular 443 port with this session ID?

$ curl -s -k https://na30.salesforce.com/services/Soap/u/41.0 \
    -H "Content-Type: text/xml; charset=UTF-8" \
    -H "SOAPAction: example" \
    -d @getuserinfo.xml
<?xml version="1.0" encoding="UTF-8"?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:sf="urn:fault.partner.soap.sforce.com" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <soapenv:Body>
    <soapenv:Fault>
      <faultcode>sf:MUTUAL_AUTHENTICATION_FAILED</faultcode>
      <faultstring>MUTUAL_AUTHENTICATION_FAILED: This session could not be mutually authenticated for use with the API</faultstring>
      <detail>
        <sf:UnexpectedErrorFault xsi:type="sf:UnexpectedErrorFault">
          <sf:exceptionCode>MUTUAL_AUTHENTICATION_FAILED</sf:exceptionCode>
          <sf:exceptionMessage>This session could not be mutually authenticated for use with the API</sf:exceptionMessage>
        </sf:UnexpectedErrorFault>
      </detail>
    </soapenv:Fault>
  </soapenv:Body>
</soapenv:Envelope>

This time we get a much more palatable response!

Now you know how to get the basics of Salesforce Mutual Authentication working. In part 2 of this series, I look at using Salesforce's Web Service Connector (WSC) to access the SOAP and Bulk APIs with Mutual Authentication, and in part 3, I explain how to access the Salesforce REST APIs with common Java HTTP clients such as the Apache and Jetty.

Tagged as: , 2 Comments