1 Accessing ESG Data Through netCDF {#esg}
2 =================================
6 Introduction {#esg_intro}
9 It is possible to access Earth Systems Grid (ESG) datasets from ESG
10 servers through the netCDF API. This requires building the netCDF
11 library with DAP2 protocol support using the "--enable-dap" flag to the
14 In order to access ESG datasets, however, it is necessary to register as
15 a user with ESG and to setup your environment so that proper
16 authentication is established between a netCDF client program and the
17 ESG data server. Specifically, it is necessary to use what is called
18 "client-side keys" to enable this authentication. Normally, when a
19 client accesses a server in a secure fashion (using "https"), the server
20 provides an authentication certificate to the client. With client-side
21 keys, the client must also provide a certificate to the server so that
22 the server can know with whom it is communicating.
24 It is possible to set up the netCDF library to use client-side keys,
25 although the process is somewhat complicated. The DAP2 support in netCDF
26 uses the *curl* library and it is that underlying library that must be
29 Terminology {#esg_term}
32 The key elements for client-side keys requires the constructions of two
33 "stores" on the client side.
35 - Keystore - a repository to hold the client side key.
36 - Truststore - a repository to hold a chain of certificates that can
37 be used to validate the certificate sent by the server to
40 The server actually has a similar set of stores, but the client need not
41 be concerned with those.
43 Initial Steps {#esg_initial_steps}
46 The first step is to obtain authorization from ESG. Note that this
47 information may evolve over time, and may be out of date. This
48 discussion is in terms of BADC ESG. You will need to substitute the ESG
49 site for BADC in the following.
51 1. Register at http://badc.nerc.ac.uk/register to obtain access to badc
52 and to obtain an openid, which will looks something like:
54 https://ceda.ac.uk/openid/Firstname.Lastname
56 2. Ask BADC for access to whatever datasets are of interest.
57 3. Obtain short term credentials at
58 http://grid.ncsa.illinois.edu/myproxy/MyProxyLogon/ You will need to
59 download and run the MyProxyLogon program. This will create a
60 keyfile in, typically, the directory globus. The keyfile will have a
61 name similar to this: x509up\_u13615 The other elements in
62 \~/.globus are certificates to use in validating the certificate
63 your client gets from the server.
64 4. Obtain the program source ImportKey.java from this location:
65 http://www.agentbob.info/agentbob/79-AB.html (read the whole page,
66 it will help you understand the remaining steps).
68 Building the KeyStore {#esg_keystore}
71 You will have to modify the keyfile in the previous step and then create
72 a keystore and install the key and a certificate. The commands are
75 openssl pkcs8 -topk8 -nocrypt -in x509up_u13615 -inform PEM -out key.der -outform DER
77 openssl x509 -in x509up_u13615 -inform PEM -out cert.der -outform DER
79 java -classpath -Dkeypassword="" -Dkeystore=./ key.der cert.der
81 Note, the file names "key.der" and "cert.der" can be whatever you
82 choose. It is probably best to leave the .der extension, though.
84 Building the TrustStore {#esg_truststore}
87 Building the truststore is a bit tricky because as provided, the
88 certificates in "globus" need some massaging. See the script below for
89 the details. The primary command is this, which is executed for every
90 certificate, c, in globus. It sticks the certificate into the file named
93 keytool -trustcacerts -storepass "password" -v -keystore "truststore" -importcert -file "${c}"
95 Running the C Client {#esg_c_client}
98 The file ".dodsrc" is used to configure curl. This file must reside
99 either in the current directory or in your home directory. It has lines
103 - \[http//x.y/\]KEY=VALUE
105 The first form provides a configuration parameter that applies to all
106 DAP2 accesses. The second form only applies to accesses to the server at
109 The following keys must be set in ".dodsrc" to support ESG access.
111 - HTTP.SSL.VALIDATE=1
112 - HTTP.COOKIEJAR=.dods\_cookies
113 - HTTP.SSL.CERTIFICATE=esgkeystore
114 - HTTP.SSL.KEY=esgkeystore
115 - HTTP.SSL.CAPATH=.globus
117 For ESG, the HTTP.SSL.CERTIFICATE and HTTP.SSL.KEY entries should have
118 same value, which is the file path for the certificate produced by
119 MyProxyLogon. The HTTP.SSL.CAPATH entry should be the path to the
120 "certificates" directory produced by MyProxyLogon.
122 Running the Java Client {#esg_java_client}
125 If you are using the Java netCDF client, then you need to add some
126 parameters to the "java" command. Specifically, add the following flags.
128 -Dkeystore="path to keystore file" -Dkeystorepassword="keystore password"
130 Script for creating Stores {#esg_script}
133 The following script shows in detail how to actually construct the key
134 and trust stores. It is specific to the format of the globus file as it
135 was when ESG support was first added. It may have changed since then, in
136 which case, you will need to seek some help in fixing this script. It
137 would help if you communicated what you changed to the author so this
138 document can be updated.
142 KEYSTORE="esgkeystore"
143 TRUSTSTORE="esgtruststore"
145 TRUSTROOT="certificates"
147 TRUSTROOTPATH="$GLOBUS/$TRUSTROOT"
148 CERTFILE="$GLOBUS/$CERT"
152 CCP="bcprov-jdk16-145.jar"
156 # Initialize needed directories
164 # Compile MyProxyCmd and ImportKey
165 javac -d ./build -classpath "$CCP" *.java
166 javac -d ./build ImportKey.java
169 java -cp "$CP myproxy.MyProxyCmd
172 openssl pkcs8 -topk8 -nocrypt -in $CERTFILE -inform PEM -out key.der -outform DER
173 openssl x509 -in $CERTFILE -inform PEM -out cert.der -outform DER
174 java -Dkeypassword=$PWD -Dkeystore=./${KEYSTORE} -cp ./build ImportKey key.der cert.der
176 # Clean up the certificates in the globus directory
177 for c in ${TRUSTROOTPATH}/*.0 ; do
178 alias=`basename $c .0`
179 sed -e '0,/---/d' <$c >/tmp/${alias}
180 echo "-----BEGIN CERTIFICATE-----" >$c
181 cat /tmp/${alias} >>$c
184 # Build the truststore
185 for c in ${TRUSTROOTPATH}/*.0 ; do
186 alias=`basename $c .0`
187 echo "adding: $TRUSTROOTPATH/${c}"
189 yes | keytool -trustcacerts -storepass "$PWD" -v -keystore ./$TRUSTSTORE -alias $alias -importcert -file "${c}"
195 Change Log {#esg_change_log}
200 - 10/17/2013 – Initial Release
202 Document Information {#esg_doc_info}
205 ------------------------------------------------------------------------
223 ------------------------------------------------------------------------