SSL Configuration - SAP RS (Sybase RS)

How to setup self-signed SSL with SAP RS

On the server-side:

1. Create an ASE certificate.
2. Enable SSL in ASE (if required).
3. Edit the interfaces/sql.ini file - add SSL definition.
4. Enable SSL in RS.
5. Restart RS.

A detailed description of the process to activate SSL for ASE

On the server-side:

If required: Installing OpenSSL on the machine.

1. Create an ASE certificate

1-7. Steps 1-7 like SSL for ASE.

If already done for your server - skip it. 
The certificates for the ASE on the server can be used also by the RS on this server.

8. Create/Copy the crt file to RS certificates folder
cd /sybvol01/sap16/OCS-16_0/bin/
cat ASAZRLNSAP16.public ASAZRLNSAP16.key > /sybvol01/sap16rs/REP-16_0/certificates/ASAZRLNSAP16.crt
File created: /sybvol01/sap16rs/REP-16_0/certificates/ASAZRLNSAP16.crt

9. Create  client certificate

9.a create ASAZRLNSAP16.txt in the RS certificates folder
cp root.crt /sybvol01/sap16rs/REP-16_0/certificates/ASAZRLNSAP16.txt

9.b add certificates content to the exists trusted.txt in the RS config folder (.../sap16rs/config/trusted.txt)

9.c copy more files to the RS certificates folder:
cp ASAZRLNSAP16.csr /sybvol01/sap16rs/REP-16_0/certificates/ASAZRLNSAP16.csr
cp ASAZRLNSAP16.key /sybvol01/sap16rs/REP-16_0/certificates/ASAZRLNSAP16.key
cp ASAZRLNSAP16.public /sybvol01/sap16rs/REP-16_0/certificates/ASAZRLNSAP16.public
cp root.crt /sybvol01/sap16rs/REP-16_0/certificates/root.crt
cp root.csr /sybvol01/sap16rs/REP-16_0/certificates/root.csr
cp root.key /sybvol01/sap16rs/REP-16_0/certificates/root.key

2. Enable SSL in ASE  (if required)

10. Enable ssl in ASE
→ if already done for the ASE (step 10) - skip it.
sp_configure "enable ssl", 1

11. Add ssl certificate into ASE (already done for ASE, step 11)

3. Edit the interfaces/sql.ini file - add SSL definition

12. Edit the interfaces/sql.ini file to create an ssl port:
ASAZRLNSAP16
    master tcp ether ASAZRLNSAP16 5000 ssl="CN=ASAZRLNSAP16"
    query tcp ether ASAZRLNSAP16 5000 ssl="CN=ASAZRLNSAP16"
 
RSFOG2 
   master tcp ether ASAZRLNSAP16 11753 ssl="CN=ASAZRLNSAP16"
   query tcp ether ASAZRLNSAP16 11753 ssl="CN=ASAZRLNSAP16"

4. Enable SSL in RS

13. Grant execute permissions for sp_serverinfo (CR# 814027)
use sybsystemprocs
go
grant execute on sp_serverinfo to public
go

14. Enable SSL on RS
1> configure replication server set use_ssl to 'on'
2> go
Config parameter 'use_ssl' is modified. This change will not take effect until the Replication Server is restarted.

5. Restart and do checks

15. Restart RS
Stop RS:
1> shutdown
2> go
Start RS:
cd /sybvol01/sap16rs/REP-16_0/install/
startserver -f RUN_RSFOG2

• Windows: stop and start the service in Services.

16. Check that SSL is enabled:

1> use RSFOG2_RSSD
2> go
1> select * from rs_config where optionname like "%ssl%"
2> go
optionname                     objid              charvalue         status comments                                                                                                                                             
 ------------------------------ ------------------ -------------
 ssl_protocol                   0x0000000000000000 TLSv1        0 Indicated the SSL protocol value of Replication Server.                                                                                                                     
 use_ssl                        0x0000000000000000 on   
should be:
1. ssl_protocol is not null
2. ssl_protocol is on

1> select @@ssl_ciphersuite
2> go
 ------------------------------
 TLS_RSA_WITH_AES_256_CBC_SHA