How to use etcd (multi-machine cluster basic mode) in Ubuntu?


Recently, I have found interesting solution. it is called as etcd. It is Distributed reliable key-value store. What is the key-value store?. It is similar with dictionary in Python. This etcd make dictionary items can be put and get on any hosts. 


1. Prerequesite


The etcd is used for cluster environment. Therefore, I will prepare 3 servers with ubuntu.


infra0     172.22.0.96 infra0.example.com

infra1     172.22.0.33 infra1.example.com

infra2     172.22.0.133 infra2.example.com 


2. Get etcd release 


I have ubuntu server. I need the etcd release for ubuntu. In this documentation. There is installation guide for Linux.


ETCD_VER=v3.3.10


# choose either URL

GOOGLE_URL=https://storage.googleapis.com/etcd

GITHUB_URL=https://github.com/etcd-io/etcd/releases/download

DOWNLOAD_URL=${GOOGLE_URL}


rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz

rm -rf /tmp/etcd-download-test


mkdir -p /tmp/etcd-download-test

curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz

tar xzvf /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz -C /tmp/etcd-download-test --strip-components=1

rm -f /tmp/etcd-${ETCD_VER}-linux-amd64.tar.gz


/tmp/etcd-download-test/etcd --version

ETCDCTL_API=3 /tmp/etcd-download-test/etcdctl version 


I will run this command on every hosts. "/tmp" directory is used in command above. However, In my case, I will change the directory "/home/ubunut"


# mkdir -p /home/ubuntu/etcd-download-test

# curl -L ${DOWNLOAD_URL}/${ETCD_VER}/etcd-${ETCD_VER}-linux-amd64.tar.gz -o 

# /home/ubuntu/etcd-${ETCD_VER}-linux-amd64.tar.gz

# tar xzvf /home/ubuntu/etcd-${ETCD_VER}-linux-amd64.tar.gz -C /home/ubuntu/etcd-download-test --strip-components=1

# rm -f /home/ubuntu/etcd-${ETCD_VER}-linux-amd64.tar.gz 


I can check the installation version with both of commands.


/home/ubuntu/etcd-download-test/etcd --version

ETCDCTL_API=3 /home/ubuntu/etcd-download-test/etcdctl version 


# /home/ubuntu/etcd-download-test/etcd --version

etcd Version: 3.3.10

Git SHA: 27fc7e2

Go Version: go1.10.4

Go OS/Arch: linux/amd64


More efficient, I will export the directory in PATH environment.


# export PATH=$PATH:/home/ubuntu/etcd-download-test


# vi /root/.bashrc
PATH=$PATH:/home/ubuntu/etcd-download-test

# source /root/.bashrc


3. Running in multi-machine cluster basic mode


I can run in single standalone mode. The command line usage is here. At first, run in single mode


etcd


I can put and get data. "ETCDCTL_API=3" is important parameter. It should be exist.


# ETCDCTL_API=3 etcdctl put mykey "this is awesome"

OK

# ETCDCTL_API=3 etcdctl get mykey
mykey
this is awesome


Also I can see the result with JSON format like below. If I want to find more example of command usage, look this documentation.


# export ETCDCTL_API=3
# etcdctl --endpoints=localhost:2379 put foo "Hello World!"
OK

# etcdctl --endpoints=localhost:2379 get foo --write-out="json"
{"header":{"cluster_id":14841639068965178418,"member_id":10276657743932975437,"revision":3,"raft_term":2},"kvs":[{"key":"Zm9v","create_revision":3,"mod_revision":3,"version":1,"value":"SGVsbG8gV29ybGQh"}],"count":1}

# etcdctl --endpoints=localhost:2379 get foo
foo
Hello World!


Even if this single mode works, but etcd is more powerful in mult-machin cluster mode. Look this documentation. To run cluster mode, there are 2 things important. IP address to listen and announce and IP address for peer hosts


etcd --name infra0 --initial-advertise-peer-urls http://172.22.0.96:2380 \

  --listen-peer-urls http://172.22.0.96:2380 \

  --listen-client-urls http://172.22.0.96:2379,http://127.0.0.1:2379 \

  --advertise-client-urls http://172.22.0.96:2379 \

  --initial-cluster-token etcd-cluster-1 \

  --initial-cluster infra0=http://172.22.0.96:2380,infra1=http://172.22.0.33:2380,infra2=http://172.22.0.133:2380 \

  --initial-cluster-state new

etcd --name infra1 --initial-advertise-peer-urls http://172.22.0.33:2380 \

  --listen-peer-urls http://172.22.0.33:2380 \

  --listen-client-urls http://172.22.0.33:2379,http://127.0.0.1:2379 \

  --advertise-client-urls http://172.22.0.33:2379 \

  --initial-cluster-token etcd-cluster-1 \

  --initial-cluster infra0=http://172.22.0.96:2380,infra1=http://172.22.0.33:2380,infra2=http://172.22.0.133:2380 \

  --initial-cluster-state new

etcd --name infra2 --initial-advertise-peer-urls http://172.22.0.133:2380 \

  --listen-peer-urls http://172.22.0.133:2380 \

  --listen-client-urls http://172.22.0.133:2379,http://127.0.0.1:2379 \

  --advertise-client-urls http://172.22.0.133:2379 \

  --initial-cluster-token etcd-cluster-1 \

  --initial-cluster infra0=http://172.22.0.96:2380,infra1=http://172.22.0.33:2380,infra2=http://172.22.0.133:2380 \

  --initial-cluster-state new

  

For peer communicate, the 2380 port is used. For data communication, the 2379 port is used. Therefore, "127.0.0.1:2379" is necessary to request to localhost. However, this is not necessary to announce. These command above should be run on each host. After running, I can get the status of clustering.


# ETCDCTL_API=3 etcdctl --endpoints=172.22.0.96:2379,172.22.0.33:2379,172.22.0.133:2379 endpoint health

172.22.0.96:2379 is healthy: successfully committed proposal: took = 2.385219ms

172.22.0.133:2379 is healthy: successfully committed proposal: took = 2.847419ms

172.22.0.33:2379 is healthy: successfully committed proposal: took = 2.292504ms


Now, I have done everything to cluster multi-hosts. In multi-hosts environments, I put data into anywhere, I can get data from anywhere.


ETCDCTL_API=3 etcdctl --endpoints=172.22.0.96:2379 put foo "Hello World!"

OK


ETCDCTL_API=3 etcdctl --endpoints=172.22.0.133:2379 get foo

foo

Hello World!


Finally, I can use the "etcd" little.


 Reference


[ 1 ] https://github.com/etcd-io/etcd

[ 2 ] https://github.com/etcd-io/etcd/releases

[ 3 ] https://github.com/etcd-io/etcd/blob/master/Documentation/op-guide/clustering.md

[ 4 ] https://coreos.com/etcd/docs/latest/demo.html

How does create Intermediate certificate with openssl?  


It is difficult to understand this intermediate certificate. I wonder why this intermediate is necessary?. I have found this documentation over Internet


The purpose of using an intermediate CA is primarily for security. The root key can be kept offline and used as infrequently as possible. If the intermediate key is compromised, the root CA can revoke the intermediate certificate and create a new intermediate cryptographic pair. 


I studied about the chain concept. I focused if the encryption with private key of root certificate and decryption with public key of child certificate are possible, reverse versa?. (The answer is "Not Possible"). The reason is to hide the root CA for security. 


1. Prerequisite


This is the default directory structure to generate intermediate certificate.


# mkdir /root/ca


# cd /root/ca

# mkdir certs crl newcerts private

# chmod 700 private

# touch index.txt

# echo 1000 > serial


To generate "Key", "CSR" (Certificate Signing Reqeust) and. CRT (Certificate), I need configuration for each case, In reference documentation, there are 2 types of configuration for Root CA and Intermediate CA. At this time, I need Root CA configuration. It looks like below.


vi /root/ca/openssl.cnf


# OpenSSL root CA configuration file.
# Copy to `/root/ca/openssl.cnf`.

[ ca ]
# `man ca`
default_ca = CA_default

[ CA_default ]
# Directory and file locations.
dir               = /root/ca
certs             = $dir/certs
crl_dir           = $dir/crl
new_certs_dir     = $dir/newcerts
database          = $dir/index.txt
serial            = $dir/serial
RANDFILE          = $dir/private/.rand

# The root key and root certificate.
private_key       = $dir/private/ca.key.pem
certificate       = $dir/certs/ca.cert.pem

# For certificate revocation lists.
crlnumber         = $dir/crlnumber
crl               = $dir/crl/ca.crl.pem
crl_extensions    = crl_ext
default_crl_days  = 30

# SHA-1 is deprecated, so use SHA-2 instead.
default_md        = sha256

name_opt          = ca_default
cert_opt          = ca_default
default_days      = 375
preserve          = no
policy            = policy_strict

[ policy_strict ]
# The root CA should only sign intermediate certificates that match.
# See the POLICY FORMAT section of `man ca`.
countryName             = match
stateOrProvinceName     = match
organizationName        = match
organizationalUnitName  = optional
commonName              = supplied
emailAddress            = optional

[ policy_loose ]
# Allow the intermediate CA to sign a more diverse range of certificates.
# See the POLICY FORMAT section of the `ca` man page.
countryName             = optional
stateOrProvinceName     = optional
localityName            = optional
organizationName        = optional
organizationalUnitName  = optional
commonName              = supplied
emailAddress            = optional

[ req ]
# Options for the `req` tool (`man req`).
default_bits        = 2048
distinguished_name  = req_distinguished_name
string_mask         = utf8only

# SHA-1 is deprecated, so use SHA-2 instead.
default_md          = sha256

# Extension to add when the -x509 option is used.
x509_extensions     = v3_ca

[ req_distinguished_name ]
# See <https://en.wikipedia.org/wiki/Certificate_signing_request>.
countryName                     = Country Name (2 letter code)
stateOrProvinceName             = State or Province Name
localityName                    = Locality Name
0.organizationName              = Organization Name
organizationalUnitName          = Organizational Unit Name
commonName                      = Common Name
emailAddress                    = Email Address

# Optionally, specify some defaults.
countryName_default             = GB
stateOrProvinceName_default     = England
localityName_default            =
0.organizationName_default      = Alice Ltd
organizationalUnitName_default  =
emailAddress_default            =

[ v3_ca ]
# Extensions for a typical CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true
keyUsage = critical, digitalSignature, cRLSign, keyCertSign

[ v3_intermediate_ca ]
# Extensions for a typical intermediate CA (`man x509v3_config`).
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid:always,issuer
basicConstraints = critical, CA:true, pathlen:0
keyUsage = critical, digitalSignature, cRLSign, keyCertSign

[ usr_cert ]
# Extensions for client certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = client, email
nsComment = "OpenSSL Generated Client Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
keyUsage = critical, nonRepudiation, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth, emailProtection

[ server_cert ]
# Extensions for server certificates (`man x509v3_config`).
basicConstraints = CA:FALSE
nsCertType = server
nsComment = "OpenSSL Generated Server Certificate"
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer:always
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth

[ crl_ext ]
# Extension for CRLs (`man x509v3_config`).
authorityKeyIdentifier=keyid:always

[ ocsp ]
# Extension for OCSP signing certificates (`man ocsp`).
basicConstraints = CA:FALSE
subjectKeyIdentifier = hash
authorityKeyIdentifier = keyid,issuer
keyUsage = critical, digitalSignature
extendedKeyUsage = critical, OCSPSigning


There are lots of options in there. I can not explain all of things. For the reproduction of this test, I check the directory path at first.


[ CA_default ]

# Directory and file locations.

dir               = /root/ca


In this post, I create "/root/ca" directory before, therefore I do not need to change this value. However, it need to be changed if I want to work on other directory.

 

2. Create root key and root certificate


I generate RSA private key which is most used with "openssl genrsa" command. (Please, note that If there is no option like "-aes256", there is no process to enter the password)


# cd /root/ca

# openssl genrsa -aes256 -out private/ca.key.pem 4096


Enter pass phrase for ca.key.pem: secretpassword

Verifying - Enter pass phrase for ca.key.pem: secretpassword


# chmod 400 private/ca.key.pem


And then, I create Root CA certificate with the Private key which is generated earlier. (In this case, I do not use the CSR). The "-x509" option is the option which is necessary when I create certificate 


# cd /root/ca

# openssl req -config openssl.cnf -key private/ca.key.pem -new -x509 -days 7300 -sha256 -extensions v3_ca -out certs/ca.cert.pem


Enter pass phrase for ca.key.pem: secretpassword

You are about to be asked to enter information that will be incorporated

into your certificate request.

-----

Country Name (2 letter code) [XX]:GB

State or Province Name []:England

Locality Name []:

Organization Name []:Alice Ltd

Organizational Unit Name []:Alice Ltd Certificate Authority

Common Name []:Alice Ltd Root CA

Email Address []:


# chmod 444 certs/ca.cert.pem


In this case, Root CA is not changed for a long time. So "-days 7300" (almost 20 years) is defined. Because of this, the Private key of Root should be secret. I can see more detail information with "verification" command. In the result, I can see the "issuer" and "validity" information. Also I can see the certificate include the Public Key Info.


# openssl x509 -noout -text -in certs/ca.cert.pem

Certificate:

    Data:

        Version: 3 (0x2)

        Serial Number: 

    Signature Algorithm: sha256WithRSAEncryption

        Issuer: C=GB, ST=England, O=Alice Ltd, OU=Alice Ltd Certificate Authority, CN=Alice Ltd Root CA

        Validity

            Not Before: Oct 19 00:53:19 2018 GMT

            Not After : Oct 14 00:53:19 2038 GMT

        Subject: C=GB, ST=England, O=Alice Ltd, OU=Alice Ltd Certificate Authority, CN=Alice Ltd Root CA

        Subject Public Key Info:

            Public Key Algorithm: rsaEncryption

                Public-Key: (4096 bit)

                Modulus:

                Exponent: 65537 (0x10001)

        X509v3 extensions:

            X509v3 Subject Key Identifier:

            X509v3 Authority Key Identifier:

            X509v3 Basic Constraints: critical

                CA:TRUE

            X509v3 Key Usage: critical

                Digital Signature, Certificate Sign, CRL Sign

    Signature Algorithm: sha256WithRSAEncryption


Therefore, I can extract the Public key.


# mkdir public


openssl x509 -pubkey -in certs/ca.cert.pem -noout > public/pub.ca.cert.pem

-----BEGIN PUBLIC KEY-----

Something ................ xxxxxxxxxxxxxxxxxxxxxxx

-----END PUBLIC KEY-----


3. Create intermediate pair


To generate the Intermediate certificate, I need to make sub-directory under the root directory.


mkdir /root/ca/intermediate


# cd /root/ca/intermediate

# mkdir certs crl csr newcerts private

# chmod 700 private

# touch index.txt

# echo 1000 > serial

# echo 1000 > /root/ca/intermediate/crlnumber


"echo 1000 > /root/ca/intermediate/crlnumber" is used for tracking certificate revocation lists. In this post, I do not handle about this. I also need configuration for intermediate certificate. Most of the contents are same with Root's configuration. 5 things are different.


[ CA_default ]

dir               = /root/ca    ==>  /root/ca/intermediate

private_key       = $dir/private/ca.key.pem    ==> $dir/private/intermediate.key.pem

certificate       = $dir/certs/ca.cert.pem    ==> $dir/certs/intermediate.cert.pem

crl               = $dir/crl/ca.crl.pem    ==> $dir/crl/intermediate.crl.pem

policy            = policy_strict    ==> policy_loose


I generate RSA Private key. this key is independent with Root Private key. This intermediate certificate is usually not used for end-point such as the server and client. I think it is kinds of sub-Root certificate. (However, this is one of cases. I can use intermediate certificate on server and client also.)


# cd /root/ca

# openssl genrsa -aes256 -out intermediate/private/intermediate.key.pem 4096


Enter pass phrase for intermediate.key.pem: secretpassword

Verifying - Enter pass phrase for intermediate.key.pem: secretpassword


# chmod 400 intermediate/private/intermediate.key.pem


Now I need to generate certificate signing request (CSR) at this time. During this step, The Common Name must be different. CSR includes several information such as "common Name". I can create certificate without this CSR. (In the Root case, I did not create CSR at that time.


# cd /root/ca

# openssl req -config intermediate/openssl.cnf -new -sha256 -key intermediate/private/intermediate.key.pem -out intermediate/csr/intermediate.csr.pem


Enter pass phrase for intermediate.key.pem: secretpassword

You are about to be asked to enter information that will be incorporated

into your certificate request.

-----

Country Name (2 letter code) [XX]:GB

State or Province Name []:England

Locality Name []:

Organization Name []:Alice Ltd

Organizational Unit Name []:Alice Ltd Certificate Authority

Common Name []:Alice Ltd Intermediate CA

Email Address []:


Now, I will generate intermediate certificate with this CSR. Please, note that I will use root certificate and configuration to create intermediate


# cd /root/ca

# openssl ca -config openssl.cnf -extensions v3_intermediate_ca -days 3650 -notext -md sha256 -in intermediate/csr/intermediate.csr.pem -out intermediate/certs/intermediate.cert.pem

Using configuration from openssl.cnf

Enter pass phrase for /root/ca/private/ca.key.pem:

Check that the request matches the signature

Signature ok

Certificate Details:

        Serial Number: 4096 (0x1000)

        Validity

            Not Before: Oct 19 01:20:29 2018 GMT

            Not After : Oct 16 01:20:29 2028 GMT

        Subject:

            countryName               = GB

            stateOrProvinceName       = England

            organizationName          = Alice Ltd

            organizationalUnitName    = Alice Ltd Certificate Authority

            commonName                = Alice Ltd Intermediate CA

        X509v3 extensions:

            X509v3 Subject Key Identifier:

            X509v3 Authority Key Identifier:

            X509v3 Basic Constraints: critical

                CA:TRUE, pathlen:0

            X509v3 Key Usage: critical

                Digital Signature, Certificate Sign, CRL Sign

Certificate is to be certified until Oct 16 01:20:29 2028 GMT (3650 days)

Sign the certificate? [y/n]:y



1 out of 1 certificate requests certified, commit? [y/n]y

Write out database with 1 new entries

Data Base Updated


# chmod 444 intermediate/certs/intermediate.cert.pem


During creation, I can see the detail information. Also, I can see mote detail with "verification" command.


# #  openssl x509 -noout -text -in intermediate/certs/intermediate.cert.pem

Certificate:

    Data:

        Version: 3 (0x2)

        Serial Number: 4096 (0x1000)

    Signature Algorithm: sha256WithRSAEncryption

        Issuer: C=GB, ST=England, O=Alice Ltd, OU=Alice Ltd Certificate Authority, CN=Alice Ltd Root CA

        Validity

            Not Before: Oct 19 01:20:29 2018 GMT

            Not After : Oct 16 01:20:29 2028 GMT

        Subject: C=GB, ST=England, O=Alice Ltd, OU=Alice Ltd Certificate Authority, CN=Alice Ltd Intermediate CA

        Subject Public Key Info:

            Public Key Algorithm: rsaEncryption

                Public-Key: (4096 bit)

                Modulus:

                Exponent: 65537 (0x10001)

        X509v3 extensions:

            X509v3 Subject Key Identifier:

            X509v3 Authority Key Identifier:

            X509v3 Basic Constraints: critical

                CA:TRUE, pathlen:0

            X509v3 Key Usage: critical

                Digital Signature, Certificate Sign, CRL Sign

    Signature Algorithm: sha256WithRSAEncryption


This certificate also include the public key, therefore I can extract the public key from certificate


# mkdir intermediate/public


# openssl x509 -pubkey -in intermediate/certs/intermediate.cert.pem -noout > intermediate/public/pub.intermediate.cert.pem

-----BEGIN PUBLIC KEY-----

Something ........ xxxxxxxxxxxxxxxxxxxxxxxx

-----END PUBLIC KEY-----



4. Verify the intermediate certificate


I obtain "Private key", "CSR" and "Certificate" files for intermediate. I have some question how does this cert guarantee?. "openssl verfify -CAfile" command show intermediate cert is certified by root certificate

 

# openssl verify -CAfile certs/ca.cert.pem intermediate/certs/intermediate.cert.pem

intermediate/certs/intermediate.cert.pem: OK


4-1. [Optional] The relationship between Root Key and Intermediate Key.


Someone can think like me, the intermediate certificate inherit from the root certification. Therefore, I can encrypt with intermediate public key and decrypt with root private key or reverse versa. The answer is "No" (RSA operation error is happend)


# echo "welecom ssl world" > message


[ Intermediate public encrypt + Root private decrypt ]

# openssl rsautl -encrypt -inkey intermediate/public/pub.intermediate.cert.pem -pubin -in message -out encrypted-message

# openssl rsautl -decrypt -inkey private/ca.key.pem -in encrypted-message -out decrypted-message

RSA operation error


[ Root public encrypt + Intermediate private decrypt ]

# openssl rsautl -encrypt -inkey public/pub.ca.cert.pem -pubin -in message -out encrypted-message

# openssl rsautl -decrypt -inkey intermediate/private/intermediate.key.pem -in encrypted-message -out decrypted-message

RSA operation error



5. Create the certificate chain file


In the documentation, there are reason why the chain file is necessary. 


When an application (eg, a web browser) tries to verify a certificate signed by the intermediate CA, it must also verify the intermediate certificate against the root certificate.  


Thus, it is necessary to confirm if the intermediate certificate come from root certificate. To create chain, I will merge both certificate into single file.


# cat intermediate/certs/intermediate.cert.pem certs/ca.cert.pem > intermediate/certs/ca-chain.cert.pem

# chmod 444 intermediate/certs/ca-chain.cert.pem


6. Sign server and client certificates


Before, I do this step. I need to know what is difference between server certificate and client certificate. Before, I write this post, I only consider web server such as Apache. In this Apache case, the server certificate is necessary. Server certificate focus if this server is safe or not. Client certificate focus if my request to the server is safe or not. So usually, client certificate is doing after server certificate. In this post, I will handle server certificate. 


I think this intermediate certificate is kinds of sub-root certificate. So this intermediate should be hidden for the security. (Don't worry, if this intermediate certificate is stolen, I can delete and re-create with Root certificate again) Anyway, I can create certificate for end-point.


# cd /root/ca

# openssl genrsa -aes256 -out intermediate/private/www.example.com.key.pem 2048

# chmod 400 intermediate/private/www.example.com.key.pem


The reason why I use 2048 bit to create RSA key. Usually, the certificate for end-point should be short expiration day. So, 2048 bit is much efficient. After this, I create the CSR with private key. And I use intermediate configuration file to generate. (At this time, it looks like another intermediate certificate). Note that the Common Name cannot be the same as either your root or intermediate certificate.


# cd /root/ca

# openssl req -config intermediate/openssl.cnf -key intermediate/private/www.example.com.key.pem -new -sha256 -out intermediate/csr/www.example.com.csr.pem


Enter pass phrase for www.example.com.key.pem: secretpassword

You are about to be asked to enter information that will be incorporated

into your certificate request.

-----

Country Name (2 letter code) [XX]:US

State or Province Name []:California

Locality Name []:Mountain View

Organization Name []:Alice Ltd

Organizational Unit Name []:Alice Ltd Web Services

Common Name []:www.example.com

Email Address []:


Now, I will create the certificate. Please note, I used "server_cert" option, because I suppose I create certificate for web-server. However if I want the certificate for client, I use "user_cert"


To create a certificate, use the intermediate CA to sign the CSR. If the certificate is going to be used on a server, use the server_cert extension. If the certificate is going to be used for user authentication, use the usr_cert extension. Certificates are usually given a validity of one year, though a CA will typically give a few days extra for convenience. 


# cd /root/ca

# openssl ca -config intermediate/openssl.cnf -extensions server_cert -days 375 -notext -md sha256 -in intermediate/csr/www.example.com.csr.pem -out intermediate/certs/www.example.com.cert.pem

# chmod 444 intermediate/certs/www.example.com.cert.pem


I can verify this certificate. At this time, I do not verify with the certificate of the intermediate. I verify with ca-chain which are generated earlier.


openssl x509 -noout -text -in intermediate/certs/www.example.com.cert.pem

openssl verify -CAfile intermediate/certs/ca-chain.cert.pem intermediate/certs/www.example.com.cert.pem


I will deploy my web server with these certificate and chain. At this time, I do not need to merge again the ca-chain.cert.pem.


ca-chain.cert.pem

www.example.com.key.pem

www.example.com.cert.pem


I compare the configuration. In "-config" option of "openssl ca", it decide the "issuer". Therefore, intermediate's issuer is root and server's issuer is intermediate. In "extension" option, it decide the rule or type of certificate. Becuase of this, the server handle another type of intermediate certificate which is issued by (sub-root) intermediate certificate.



Therefore, the certification does not need to merge at this time. I do test again to check the relationship between intermediate certifications. In the result, I can not encrypt and decrypt. 


# openssl rsautl -encrypt -inkey intermediate/public/pub.www.example.com.cert.pem -pubin -in message -out encrypt-message

# openssl rsautl -decrypt -inkey intermediate/private/intermediate.key.pem -in encrypt-message -out decrypt-message

RSA operation error




Reference 


[ 1 ] https://jamielinux.com/docs/openssl-certificate-authority/index.html#openssl-certificate-authority

[ 2 ] https://jamielinux.com/docs/openssl-certificate-authority/appendix/root-configuration-file.html

[ 3 ] https://www.websecurity.symantec.com/security-topics/client-certificates-vs-server-certificates

How to use ECDSA?


Recently, I have some changes to know about "ECDSA" to signature the messages. I am not the security engineer. So, I can not explain all of things about this. Before, I read about this. I only think about the encryption and decryption. However, "ECDSA" shows another mechanism. Most of the documentations, "ECDSA" uses to sign and verfication. Let's think about RSA case, the RSA focus on "How can the file encrypt and decrypt". Therefore, encrypted-file is created and transferred to the destination. However, the ECDSA case is different. the ECDSA focus on "How can this message confirm to be same with others". Therefore, no encrypted-file is created, but the signature is generated. The destination has already known the message, it is verified with this signature. In this post, I will use "openssl" mainly. 


1. Create the Private key


In ECDSA algorithm, the concept of the private and public key. I can create a private key at first. With the private key, I can generate the signature for the some messages.


# openssl ecparam -genkey -name sect571r1 -out private.pem


Above, there is "name" field which has already defined. With "-list_curves" option, I can check what name is possible. In my case, "sect571r1" is one what I choose.


# openssl ecparam -list_curves

  secp112r1 : SECG/WTLS curve over a 112 bit prime field

  secp112r2 : SECG curve over a 112 bit prime field

  secp128r1 : SECG curve over a 128 bit prime field

  secp128r2 : SECG curve over a 128 bit prime field

  secp160k1 : SECG curve over a 160 bit prime field

  secp160r1 : SECG curve over a 160 bit prime field

  secp160r2 : SECG/WTLS curve over a 160 bit prime field

  secp192k1 : SECG curve over a 192 bit prime field

  secp224k1 : SECG curve over a 224 bit prime field

  secp224r1 : NIST/SECG curve over a 224 bit prime field

  secp256k1 : SECG curve over a 256 bit prime field

  secp384r1 : NIST/SECG curve over a 384 bit prime field

  secp521r1 : NIST/SECG curve over a 521 bit prime field

  prime192v1: NIST/X9.62/SECG curve over a 192 bit prime field

  prime192v2: X9.62 curve over a 192 bit prime field

  prime192v3: X9.62 curve over a 192 bit prime field

  prime239v1: X9.62 curve over a 239 bit prime field

  prime239v2: X9.62 curve over a 239 bit prime field

  prime239v3: X9.62 curve over a 239 bit prime field

  prime256v1: X9.62/SECG curve over a 256 bit prime field

  sect113r1 : SECG curve over a 113 bit binary field

  sect113r2 : SECG curve over a 113 bit binary field

  sect131r1 : SECG/WTLS curve over a 131 bit binary field

  sect131r2 : SECG curve over a 131 bit binary field

  sect163k1 : NIST/SECG/WTLS curve over a 163 bit binary field

  sect163r1 : SECG curve over a 163 bit binary field

  sect163r2 : NIST/SECG curve over a 163 bit binary field

  sect193r1 : SECG curve over a 193 bit binary field

  sect193r2 : SECG curve over a 193 bit binary field

  sect233k1 : NIST/SECG/WTLS curve over a 233 bit binary field

  sect233r1 : NIST/SECG/WTLS curve over a 233 bit binary field

  sect239k1 : SECG curve over a 239 bit binary field

  sect283k1 : NIST/SECG curve over a 283 bit binary field

  sect283r1 : NIST/SECG curve over a 283 bit binary field

  sect409k1 : NIST/SECG curve over a 409 bit binary field

  sect409r1 : NIST/SECG curve over a 409 bit binary field

  sect571k1 : NIST/SECG curve over a 571 bit binary field

  sect571r1 : NIST/SECG curve over a 571 bit binary field

  c2pnb163v1: X9.62 curve over a 163 bit binary field

  c2pnb163v2: X9.62 curve over a 163 bit binary field

  c2pnb163v3: X9.62 curve over a 163 bit binary field

  c2pnb176v1: X9.62 curve over a 176 bit binary field

  c2tnb191v1: X9.62 curve over a 191 bit binary field

  c2tnb191v2: X9.62 curve over a 191 bit binary field

  c2tnb191v3: X9.62 curve over a 191 bit binary field

  c2pnb208w1: X9.62 curve over a 208 bit binary field

  c2tnb239v1: X9.62 curve over a 239 bit binary field

  c2tnb239v2: X9.62 curve over a 239 bit binary field

  c2tnb239v3: X9.62 curve over a 239 bit binary field

  c2pnb272w1: X9.62 curve over a 272 bit binary field

  c2pnb304w1: X9.62 curve over a 304 bit binary field

  c2tnb359v1: X9.62 curve over a 359 bit binary field

  c2pnb368w1: X9.62 curve over a 368 bit binary field

  c2tnb431r1: X9.62 curve over a 431 bit binary field

  wap-wsg-idm-ecid-wtls1: WTLS curve over a 113 bit binary field

  wap-wsg-idm-ecid-wtls3: NIST/SECG/WTLS curve over a 163 bit binary field

  wap-wsg-idm-ecid-wtls4: SECG curve over a 113 bit binary field

  wap-wsg-idm-ecid-wtls5: X9.62 curve over a 163 bit binary field

  wap-wsg-idm-ecid-wtls6: SECG/WTLS curve over a 112 bit prime field

  wap-wsg-idm-ecid-wtls7: SECG/WTLS curve over a 160 bit prime field

  wap-wsg-idm-ecid-wtls8: WTLS curve over a 112 bit prime field

  wap-wsg-idm-ecid-wtls9: WTLS curve over a 160 bit prime field

  wap-wsg-idm-ecid-wtls10: NIST/SECG/WTLS curve over a 233 bit binary field

  wap-wsg-idm-ecid-wtls11: NIST/SECG/WTLS curve over a 233 bit binary field

  wap-wsg-idm-ecid-wtls12: WTLS curvs over a 224 bit prime field

  Oakley-EC2N-3:

        IPSec/IKE/Oakley curve #3 over a 155 bit binary field.

        Not suitable for ECDSA.

        Questionable extension field!

  Oakley-EC2N-4:

        IPSec/IKE/Oakley curve #4 over a 185 bit binary field.

        Not suitable for ECDSA.

        Questionable extension field!

  brainpoolP160r1: RFC 5639 curve over a 160 bit prime field

  brainpoolP160t1: RFC 5639 curve over a 160 bit prime field

  brainpoolP192r1: RFC 5639 curve over a 192 bit prime field

  brainpoolP192t1: RFC 5639 curve over a 192 bit prime field

  brainpoolP224r1: RFC 5639 curve over a 224 bit prime field

  brainpoolP224t1: RFC 5639 curve over a 224 bit prime field

  brainpoolP256r1: RFC 5639 curve over a 256 bit prime field

  brainpoolP256t1: RFC 5639 curve over a 256 bit prime field

  brainpoolP320r1: RFC 5639 curve over a 320 bit prime field

  brainpoolP320t1: RFC 5639 curve over a 320 bit prime field

  brainpoolP384r1: RFC 5639 curve over a 384 bit prime field

  brainpoolP384t1: RFC 5639 curve over a 384 bit prime field

  brainpoolP512r1: RFC 5639 curve over a 512 bit prime field

  brainpoolP512t1: RFC 5639 curve over a 512 bit prime field


2. Create the Public key


Now, I can create the public key with private key.


# openssl ec -in private.pem -pubout -out public.pem

read EC key

writing EC key


3. Sign the message


Before sign the message, I create message file like below.


# cat message

this is test message to check ecdsa with openssl


Now, I sign this message file with "openssl dgst" command.


openssl dgst -sha1 -sign private.pem message > message-sign.bin


In this case, I choose the "sha1" hashing algorithm. (Default, sha256) However, I can choose with more options.


# openssl dgst --help

unknown option '--help'

options are

-c              to output the digest with separating colons

-r              to output the digest in coreutils format

-d              to output debug info

-hex            output as hex dump

-binary         output in binary form

-hmac arg       set the HMAC key to arg

-non-fips-allow allow use of non FIPS digest

-sign   file    sign digest using private key in file

-verify file    verify a signature using public key in file

-prverify file  verify a signature using private key in file

-keyform arg    key file format (PEM or ENGINE)

-out filename   output to filename rather than stdout

-signature file signature to verify

-sigopt nm:v    signature parameter

-hmac key       create hashed MAC with key

-mac algorithm  create MAC (not neccessarily HMAC)

-macopt nm:v    MAC algorithm parameters or key

-engine e       use engine e, possibly a hardware device.

-md4            to use the md4 message digest algorithm

-md5            to use the md5 message digest algorithm

-ripemd160      to use the ripemd160 message digest algorithm

-sha            to use the sha message digest algorithm

-sha1           to use the sha1 message digest algorithm

-sha224         to use the sha224 message digest algorithm

-sha256         to use the sha256 message digest algorithm

-sha384         to use the sha384 message digest algorithm

-sha512         to use the sha512 message digest algorithm

-whirlpool      to use the whirlpool message digest algorithm


4. Verify the message 


I got some signature for message. Now I can my message which I have already gotten with this signature.


# cp message received-message


# openssl dgst -sha1 -verify public.pem -signature message-sign.bin received-message

Verified OK


If I change the contents in received-message file like below.


cat received-message

this is test message to check ecdsa with openssl

more information


# openssl dgst -sha1 -verify public.pem -signature message-sign.bin received-message
Verification Failure


I will get the verification failure


Reference


[ 1 ] https://superuser.com/questions/737574/openssl-ecdsa-sign-and-verify-file

[ 2 ] https://www.openssl.org/docs/man1.0.2/apps/ecparam.html

[ 3 ] https://wiki.openssl.org/index.php/Command_Line_Elliptic_Curve_Operations

How to use the public, private key-pair and certificate?



I can meet lots of SSL certification to protect web server host. I am not the security engineer. Therefore, It is difficult to understand the relationship private-public key pair and the certification. I have recently found the answer from here. Now I will follow these if it works or not. In this post, I will use "openssl" to handle.


1. Install the "openssl" on ubuntu


Basically, this openssl has been installed on ubuntu, therfore, I do not need to install again.


# apt-get install openssl 


2. Generate the private keys


In this post, I create private key of 2048 size with RSA algorithm at first. 


# openssl genrsa -out myprivate.pem 2048

Generating RSA private key, 2048 bit long modulus

......+++

...............................................+++

e is 65537 (0x10001)


# cat myprivate.pem

-----BEGIN RSA PRIVATE KEY-----

something............xxxxxxxxxxxxxxxxxxxxxxxxx

-----END RSA PRIVATE KEY----- 


3. Generate the public keys


With the private key, I can generate the public key with RSA key management command.


# openssl rsa -in myprivate.pem -outform PEM -pubout -out public.pem

writing RSA key


# cat public.pem

-----BEGIN PUBLIC KEY-----

something............xxxxxxxxxxxxxxxxxxxxxxxxx

-----END PUBLIC KEY-----


4. Create a CSR (Certificate Signing Request)


To create a CSR, "req" command is for PKCS#10 X.509 Certificate Signing Request (CSR) Management. CSR should be created with the private key which is created. During creation, some information are required to insert.


# openssl req -new -key myprivate.pem -out mycert.csr

You are about to be asked to enter information that will be incorporated

into your certificate request.

What you are about to enter is what is called a Distinguished Name or a DN.

There are quite a few fields but you can leave some blank

For some fields there will be a default value,

If you enter '.', the field will be left blank.

-----

Country Name (2 letter code) [AU]:KR

State or Province Name (full name) [Some-State]:SEOUL

Locality Name (eg, city) []:

Organization Name (eg, company) [Internet Widgits Pty Ltd]:

Organizational Unit Name (eg, section) []:

Common Name (e.g. server FQDN or YOUR name) []:

Email Address []:


Please enter the following 'extra' attributes

to be sent with your certificate request

A challenge password []:

An optional company name []:


After above status, I will get csr file.


5. Create a Self-signed Certificate 


With the create CSR file and Private key, I can create the Self-signed Certificate (CRT) file, which I can share

# openssl x509 -req -days 365 -in mycert.csr -signkey myprivate.pem -out cert.crt

Signature ok

subject=/C=KR/ST=SEOUL/O=Internet Widgits Pty Ltd

Getting Private key


I create the CRT file with expiration. (-days option define the date to expire). This is important because CRT file can be shared.


6. How to Use these Keys and Certifications


Now, I have four files (Public-Private Key Pairs, CSR and CRT files). At first, I create sample documentation. 


# cat this_sample.txt

Hi, I am doing some test now


Before I encrypt file above. I want to see the command option of "openssl rsautl". In this command, I will use -encrypt and -decrypt options. Please note that -encrypt require "public key" not "private key" and reverse versa. "-pkcs" option is default pandding option.


# openssl rsautl --help

Usage: rsautl [options]

-in file        input file

-out file       output file

-inkey file     input key

-keyform arg    private key format - default PEM

-pubin          input is an RSA public

-certin         input is a certificate carrying an RSA public key

-ssl            use SSL v2 padding

-raw            use no padding

-pkcs           use PKCS#1 v1.5 padding (default)

-oaep           use PKCS#1 OAEP

-sign           sign with private key

-verify         verify with public key

-encrypt        encrypt with public key

-decrypt        decrypt with private key

-hexdump        hex dump output

-engine e       use engine e, possibly a hardware device.

-passin arg    pass phrase source


Now, I encrypt this file with Public key with RSA utility command which are used for signing, verification, encryption and decryption. Please, note "-pubin" option is important factor to encrypt file.


# openssl rsautl -encrypt -inkey public.pem -pubin -in this_sample.txt -out encrypted_sample


Now, I have encrypted file. At this time, I have some question. How can I recover this file. 


# openssl rsautl -decrypt -inkey myprivate.pem -keyform PEM -in encrypted_sample -out decrypted_sample


# cat decrypted_sample

Hi, I am doing some test now


It's works. However, I have something left. What is the CRT file for?. CRT file can be shared. Someone can get the public key from this CRT file.


# openssl x509 -pubkey -in cert.crt -out certpubkey.pem

-----BEGIN PUBLIC KEY-----

something......... xxxxxxxxxxxxxxxxxxx

-----END PUBLIC KEY-----


With this public key, I can send some file with encryption.


7. Encrypt with Private Key and Decrypt with Public Key


So far, I encrypt with Public key and I decrypt with Private key. However, I have question if it is do with reverse. The answer is "yes". However, it is not possible with "openssl" command line. Therefore, I can not handle this anymore at this time in this post.


Reference


[ 1 ] https://security.stackexchange.com/questions/108508/how-do-i-produce-a-ca-signed-public-key

[ 2 ] https://unix.stackexchange.com/questions/296697/how-to-encrypt-a-file-with-private-key

What to use WMI(Windows Management Instrumentation) from remote Linux server?



Usually, I can access and run some command with SSH. In this case, I used "Paramiko" module. This is sample code which I create. However, I have some questions from here. It is if is possible to access and run some command with RDP. At this time. I can not found any solution for this. But I have found about "WMI (Windows Management Instrucmentation)". It make I can get some information which is offered by Window OS with WMI.


1. Install python-wmi-client-wrapper.


In this "Git", there is the way how to installation for this


pip install wmi-client-wrapper 


2. Install WMI


"WMI" is the package for Window OS. Therefore, the wrapper is necessary. The first step is to obtain these wrapper. Thus, I still need "WMI" main module. In my case, I used Ubuntu 16.04 LTS, and I will install WMI 1.3.16.


ulimit -n 100000

cd /tmp

mkdir wmic

cd wmic


apt install autoconf gcc libdatetime-perl make build-essential g++ python-dev

wget http://www.opsview.com/sites/default/files/wmi-1.3.16.tar_.bz2

bunzip2 wmi-1.3.16.tar_.bz2

tar -xvf wmi-1.3.16.tar_

cd wmi-1.3.16/


After above, I need edit some line of file to make and build this WMI.


vim Samba/source/pidl/pidl

:583 (to jump to line 583)

remove the word defined before @$pidl

:wq

========= Look here =============
$pidl = Parse::Pidl::IDL::parse_file($idl_file, \@opt_incdirs);
defined @$pidl || die "Failed to parse $idl_file";  >>>> @$pidl || die "Failed to parse $idl_file";
require Parse::Pidl::Typelist;
=============================


And I need export some values also.


export ZENHOME=/usr

make "CPP=gcc -E -ffreestanding"

cp Samba/source/bin/wmic /bin


Now, I can use WMI and WMI-wrapper. From now, I will do some sample code. 


3. Create the sample code.


I will create some sample code to obtain the Window Processor Information.


import wmi_client_wrapper as wmi


wmic = wmi.WmiClientWrapper(

    username="Administrator",

    password="password",

    host="172.22.0.123",

)


output = wmic.query("SELECT * FROM Win32_Processor")


After run this command, I will get some result with JSON format.


# ./sample.py

[{'L2CacheSize': '0', 'VMMonitorModeExtensions': False, 'ConfigManagerErrorCode': '0', 'VoltageCaps': '0', 'PowerManagementSupported': False, 'LoadPercentage': '12', 'SerialNumber': '', 'ThreadCount': '0', 'Version': '', 'MaxClockSpeed': '2400', 'CpuStatus': '1', 'PartNumber': '', 'SecondLevelAddressTranslationExtensions': False, 'Revision': '16130', 'Status': 'OK', 'PNPDeviceID': None, 'L2CacheSpeed': '0', 'AddressWidth': '64', 'ConfigManagerUserConfig': False, 'ErrorCleared': False, 'ProcessorId': '1789FBFF000306F2', 'ProcessorType': '3', 'DeviceID': 'CPU0', 'CurrentVoltage': '0', 'CurrentClockSpeed': '2400', 'Manufacturer': 'GenuineIntel', 'Name': 'Intel(R) Xeon(R) CPU E5-2676 v3 @ 2.40GHz', 'InstallDate': None, 'Level': '6', 'AssetTag': '', 'SocketDesignation': 'CPU 1', 'NumberOfCores': '1', 'Caption': 'Intel64 Family 6 Model 63 Stepping 2', 'StatusInfo': '3', 'Architecture': '9', 'UniqueId': None, 'PowerManagementCapabilities': 'NULL', 'OtherFamilyDescription': None, 'Description': 'Intel64 Family 6 Model 63 Stepping 2', 'CreationClassName': 'Win32_Processor', 'NumberOfLogicalProcessors': '1', 'Family': '5', 'ErrorDescription': None, 'Characteristics': '2816', 'UpgradeMethod': '1', 'SystemName': 'EC2AMAZ-JC32MSV', 'NumberOfEnabledCore': '108', 'LastErrorCode': '0', 'ExtClock': '0', 'Stepping': None, 'VirtualizationFirmwareEnabled': False, 'Role': 'CPU', 'L3CacheSize': '0', 'L3CacheSpeed': '0', 'Availability': '3', 'SystemCreationClassName': 'Win32_ComputerSystem', 'DataWidth': '64'}]


4. Troubleshooting


During I try this, I do not open any security rule for this. I am question for this. I want to know if what port is used for this running. I dump the packet on my host. "135" Port are used. This port is RPC for Window. It is default opened.


04:19:44.298786 IP 172.22.0.216.46372 > 172.22.0.123.135: Flags [S], seq 2010311507, win 26883, options [mss 8961,sackOK,TS val 2762852 ecr 0,nop,wscale 7], length 0

04:19:44.299089 IP 172.22.0.123.135 > 172.22.0.216.46372: Flags [S.], seq 1265146297, ack 2010311508, win 8192, options [mss 8961,nop,w cale 8,sackOK,TS val 1122704139 ecr 2762852], length 0

04:19:44.299098 IP 172.22.0.216.46372 > 172.22.0.123.135: Flags [.], ack 1, win 211, options [nop,nop,TS val 2762853 ecr 1122704139], length 0


I run "netstat -an" on Window. The result is look like below.


  TCP    172.22.0.123:59796     52.23.123.168:443      ESTABLISHED

  TCP    172.22.0.123:60018     198.252.206.25:443     ESTABLISHED

  TCP    [::]:135               [::]:0                 LISTENING

  TCP    [::]:445               [::]:0                 LISTENING 


Reference 


[ 1 ] https://www.shellandco.net/wmic-command-ubuntu-16-04-lts/

[ 2 ] https://askubuntu.com/questions/885407/installing-wmic-on-ubuntu-16-04-lts

[ 3 ] https://github.com/kanzure/python-wmi-client-wrapper



+ Recent posts