SSL Configuration - SAP ASE (Sybase)

How to setup self-signed SSL with ASE

On the server-side:

1. Create an ASE certificate.
2. Enable SSL in ASE.
3. Edit the interfaces/sql.ini file - add SSL definition.
4. Restart ASE.

On the clients:

1. Copy the certificate files to C:\SAP\ini in the client.
2. Edit sql.ini / interface file - add SSL definition.
3. Restart ASE

The process based on https://launchpad.support.sap.com/#/notes/0001899365

A detailed description of the process to activate SSL for ASE

On the server-side:

If required: Installing OpenSSL on the machine.

1. Create an ASE certificate

1. cd /sybvol01/sap16/OCS-16_0/bin/

2. Create root certificate
openssl genrsa -passout pass:dba4ever -out root.key 4096
output:
Generating RSA private key, 4096 bit long modulus (2 primes)
.............................................................................++++
.................................++++
e is 65537 (0x010001)
File created: root.key

3. Create root certificate request to be signed:
openssl req -new -key root.key -passin pass:dba4ever -out root.csr -subj "/C=XX/ST=XX/L=city/O=Org/CN=root"
File created root.csr

4. Self-Sign root certificate
openssl x509 -req -days 3650 -in root.csr -signkey root.key -passin pass:dba4ever -out root.crt
    output:
    Signature ok
    subject=C = XX, ST = XX, L = city, O = Org, CN = root
    Getting Private key

File created: root.crt

5. Create ASE private key (Note: Add your servername/"select @@servername" in for ASEname):
openssl genrsa -des3 -passout pass:dba4ever -out ASAZRLNSAP16.key 2048
File created: ASAZRLNSAP16.key

6. Create ASE certificate request to be signed:
openssl req -new -key ASAZRLNSAP16.key -passin pass:dba4ever -out ASAZRLNSAP16.csr -subj "/C=XX/ST=XX/L=city/O=Orig/CN=ASAZRLNSAP16"
File created: ASAZRLNSAP16.csr

7. Sign ASE request with root:
openssl x509 -req -days 3650 -in ASAZRLNSAP16.csr -CA root.crt -CAkey root.key -passin pass:dba4ever -set_serial 1 -out ASAZRLNSAP16.public
    output:
    Signature ok
    subject=C = XX, ST = XX, L = city, O = Orig, CN = ASAZRLNSAP16
    Getting CA Private Key
File created: ASAZRLNSAP16.public

8. Combine ASE certificates together:
cat ASAZRLNSAP16.public ASAZRLNSAP16.key > $SYBASE/$SYBASE_ASE/certificates/ASAZRLNSAP16.crt
File created: ASAZRLNSAP16.crt

9. Create ASE client certificate.
This is the root public certificate created in step 3. This is for all client side connections.

cp root.crt $SYBASE/$SYBASE_ASE/certificates/ASAZRLNSAP16.txt

cp root.crt $SYBASE/config/trusted.txt
File created:
    $SYBASE/$SYBASE_ASE/certificates/ASAZRLNSAP16.txt
    $SYBASE/config/trusted.txt

content of file:
-----BEGIN CERTIFICATE-----
MIIFEzCCAvsCFCi6ELH5OlZt96P56Zkz3r+DUeWJMA0GCSqGSIb3DQEBCwUAMEYx
CzAJBgNVBAYTAlhYMQswCQYDVQQIDAJYWDENMAsGA1UEBwwEY2l0eTEMMAoGA1UE
CgwDT3JnMQ0wCwYDVQQDDARyb290MB4XDTIwMDYxNjE0NTEwNFoXDTMwMDYxNDE0
NTEwNFowRjELMAkGA1UEBhMCWFgxCzAJBgNVBAgMAlhYMQ0wCwYDVQQHDARjaXR5
MQwwCgYDVQQKDANPcmcxDTALBgNVBAMMBHJvb3QwggIiMA0GCSqGSIb3DQEBAQUA
A4ICDwAwggIKAoICAQDD7h2H+PPYXlt+E9mvxFTnayo2S/2+TNLwpXYlrnasbbzG
hT9c2y6FD/NMo7C7ncrLHr+BUM5cBNRfDyijeb8Tm1ASQiSCj/CJQYDbxJ/VBt4N
aESN8POOMbNqQezZISWk8dPX4cHOh8d5oZqAUo/D+Y69VScAOKIGq1PmT1a/StXz
TIiw+xmibmvBClRC9oH9vujVFDTOG8fwUm38yaV9iz30upriPL9Ly17/klzauaRY
yJ4N6HRC79jZwgbMTbHP3j5fPsETstCD9S7LZxUNmlyXW14bndU/EYjeKGHUzvtr
7mIINQTc7u1/dRZGQVGl7NtaP5t0MIH7I5e1H8y56wQHd8umaE+Uwv+P9Mn2J9bb
zRIVcK/6vVz5KdU3XahgEr8U0mbA93Kn/8Xijx1wbzTFYcvqLpNd2MGSWEjy08N9
kxN13tC/aTm8vzkygnUvrfzjeYXXsji8fRDMYCanYzutA2FWpm+L0im8ob8uqwoL
egB+ZGOkuIwKrqP0gPJXr2GFTqgRyOWbfTHC31mv0Qo3Rg5bfsPXUkIbSBXUrz6i
pKYsMjFSi3npmuw/CCli2RIQMtCMMv8kn2Q0WqlWvePXNBcEiwVs1f/8J3h/cXAN
fjU2iaqT2aANc40hrmmJPVR/L5AladwGRESzphFEacVoKz/c+RZP/0wm2EQzMQID
AQABMA0GCSqGSIb3DQEBCwUAA4ICAQATVj1EazHcTonBVcthL2mgQaluSJaMYnxi
tEy8UKWBJj4Bu+THed7FCwJy9MDw5ReZgB1yh+PQvclg19LbGrVX+x9W/3cuQ9Cy
A7PMl9r3B5WnNwzCvxlltDenzzszcOGXSP+oUAql8nYp6wM9FbLnjW3aZ036d5+V
QEx8xsbEZ+9Qwb7SFWkIJky8sEVjueeXb9u0WP84bYrSp+T+YOESWUwZvwvJGCwc
YsRM1nCpLgFQlQNZjjtPcx/lKSB0+gKGmlrePoyaa8MYlswRanzYdnk487dAk3u9
X9JYO2yyNL7drGBf0VVnT8b7X2nOlDnw0wYs4mgIuLwBWhSbYpNHYcPFCxXjEoh1
jjdGnZGsQSGMoPKfxpNq8sZUyhzEroJHpV7OxLikvpRSz/IUhLpyNWoHVKaAehkJ
Iwc9pDja4Fz2ArO5l5g7P60rwq2cCTS0zKXrRqaPGm11WIX6ovCpobOZwBDvRGiv
6KiTF2y5Ib9xL+9Q6E8R0kPWOsPzOLaeyN5pz4IHLbRPppCIAurWxD4bozJ5CWVP
W9l+LF3IIxoLLtK2+8lxHiqMiV9o9PJTBd0I83YjPJ+QrB6ARMIg2fJWAaVQH8CN
Mdlw0aVwiVo9X06Lb6FW4AC9a2b5iBVF+ncUPeVIZYyrHNmoA65sEy/hb2rwdS+w
FoueokT6Zg==
-----END CERTIFICATE-----

2. Enable SSL in ASE

10. Enable ssl in ASE:
sp_configure "enable ssl", 1;

11. Add ssl certificate into ASE:
Note: Use fully qualified path

sp_ssladmin addcert, "/sybvol01/sap16/ASE-16_0/certificates/ASAZRLNSAP16.crt", "dba4ever"

3. Edit the interfaces/sql.ini file - add SSL definition

12. Edit the interfaces/sql.ini file to create an ssl port:
   interfaces (Unix)
   -------------
    ASEname
           master tcp ether myhost myport  ssl
           query tcp ether myhost myport ssl        
    sql.ini  (Windows)
    -------
    [ASEname]
    master=tcp,myhost,myport,ssl
    query=tcp,myhost,myport,ssl

for example:
ASAZRLNSAP16
  master tcp ether asazrlnsap16 5000 ssl="CN=ASAZRLNSAP16"
  query tcp ether asazrlnsap16 5000 ssl="CN=ASAZRLNSAP16"

4. Restart and do checks

13. Restart your ASE.

14. Checks

14.1 Check the log to make sure everything loaded:
16.0:
kernel  Common Crypto Library SSL symbols loaded.
kernel  Common Crypto Library SSL startup succeeded.
...
kernel network name host, interface IPv4, address ipaddress, type ssltcp, port port, filter ssl
15.7:
kernel Certificate load from file `$SYBASE/$SYBASE_ASE/certificates/ASEname.crt`: succeeded.
kernel Trusted root certificates loaded from file '$SYBASE/$SYBASE_ASE/certificates/ASEname.txt': succeeded.
For all versions:
kernel network name host, interface IPv4, address ipaddress, type ssltcp, port port, filter ssl

14.2 sp_ssladmin lscert
1> sp_ssladmin lscert
2> go
 certificate_path
 ---------------------------------------------------------
 /sybvol01/sap16/ASE-16_0/certificates/ASAZRLNSAP16.crt 

14.3 Check that "select @@ssl_ciphersuite" return value
1> select @@ssl_ciphersuite
2> go
 ----------------------------------
 TLS_RSA_WITH_AES_256_CBC_SHA 


On the clients:

1. Copy the certificate files to C:\SAP\ini in the client.

1. copy ASAZRLNSAP16.crt and ASAZRLNSAP16.txt to C:\SAP\ini in the client.

2. Edit sql.ini / interface file - add SSL definition.

2. Edit sql.ini / interface file - add SSL definition

3. Restart ASE

3. Restart ASE