Encryption madness. Part II

Do you remember a story about dfc.crypto.repository parameter? Below is a continuation.

Case: I want to perform upgrade from 6.7SP2 to 7.2. What should I expect? Upgrade procedure will fail because AES became a standard in 2001, but it seems that EMC got to know about that only in 2013 and decided to break upgrade procedure:

- We happy?
- Vincent! We happy?
- Yeah, we happy.

Don’t you see anything strange? I do not understand what “crypto_mode = AES128_RSA1024_SHA256” and “crypto_mode = 3DES_RSA1024_SHA256” parameters do mean, actually, I do know what the abbreviations “AES”, “RSA”, “DES” and “SHA” do mean, but I have no idea what do they mean combined together. So, let’s check documentation:

Cool, now I know even less than I knew 5 minutes ago šŸ˜¦ What is my problem? I’m too smartcurious, and I do know that 3DES and AES are symmetric ciphers, RSA is a asymmetric encryption algorithm which is primarily used for key exchange, and SHA is a MAC algorithm (message authentication code). And being brought together all these abbreviations make sense only for SSL/TLS, let me demonstrate:

Andreys-MacBook-Pro:~ apanfilov$ openssl ciphers -v
DHE-RSA-AES256-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(256)  Mac=SHA1
DHE-DSS-AES256-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(256)  Mac=SHA1
AES256-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(256)  Mac=SHA1
EDH-RSA-DES-CBC3-SHA    SSLv3 Kx=DH       Au=RSA  Enc=3DES(168) Mac=SHA1
EDH-DSS-DES-CBC3-SHA    SSLv3 Kx=DH       Au=DSS  Enc=3DES(168) Mac=SHA1
DES-CBC3-SHA            SSLv3 Kx=RSA      Au=RSA  Enc=3DES(168) Mac=SHA1
DES-CBC3-MD5            SSLv2 Kx=RSA      Au=RSA  Enc=3DES(168) Mac=MD5 
DHE-RSA-AES128-SHA      SSLv3 Kx=DH       Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-DSS-AES128-SHA      SSLv3 Kx=DH       Au=DSS  Enc=AES(128)  Mac=SHA1
AES128-SHA              SSLv3 Kx=RSA      Au=RSA  Enc=AES(128)  Mac=SHA1
DHE-RSA-SEED-SHA        SSLv3 Kx=DH       Au=RSA  Enc=SEED(128) Mac=SHA1
DHE-DSS-SEED-SHA        SSLv3 Kx=DH       Au=DSS  Enc=SEED(128) Mac=SHA1
SEED-SHA                SSLv3 Kx=RSA      Au=RSA  Enc=SEED(128) Mac=SHA1
RC2-CBC-MD5             SSLv2 Kx=RSA      Au=RSA  Enc=RC2(128)  Mac=MD5 
RC4-SHA                 SSLv3 Kx=RSA      Au=RSA  Enc=RC4(128)  Mac=SHA1
RC4-MD5                 SSLv3 Kx=RSA      Au=RSA  Enc=RC4(128)  Mac=MD5 
RC4-MD5                 SSLv2 Kx=RSA      Au=RSA  Enc=RC4(128)  Mac=MD5 
EDH-RSA-DES-CBC-SHA     SSLv3 Kx=DH       Au=RSA  Enc=DES(56)   Mac=SHA1
EDH-DSS-DES-CBC-SHA     SSLv3 Kx=DH       Au=DSS  Enc=DES(56)   Mac=SHA1
DES-CBC-SHA             SSLv3 Kx=RSA      Au=RSA  Enc=DES(56)   Mac=SHA1
DES-CBC-MD5             SSLv2 Kx=RSA      Au=RSA  Enc=DES(56)   Mac=MD5 
EXP-EDH-RSA-DES-CBC-SHA SSLv3 Kx=DH(512)  Au=RSA  Enc=DES(40)   Mac=SHA1 export
EXP-EDH-DSS-DES-CBC-SHA SSLv3 Kx=DH(512)  Au=DSS  Enc=DES(40)   Mac=SHA1 export
EXP-DES-CBC-SHA         SSLv3 Kx=RSA(512) Au=RSA  Enc=DES(40)   Mac=SHA1 export
EXP-RC2-CBC-MD5         SSLv3 Kx=RSA(512) Au=RSA  Enc=RC2(40)   Mac=MD5  export
EXP-RC2-CBC-MD5         SSLv2 Kx=RSA(512) Au=RSA  Enc=RC2(40)   Mac=MD5  export
EXP-RC4-MD5             SSLv3 Kx=RSA(512) Au=RSA  Enc=RC4(40)   Mac=MD5  export
EXP-RC4-MD5             SSLv2 Kx=RSA(512) Au=RSA  Enc=RC4(40)   Mac=MD5  export

Doesn’t it look familiar? So, how does the crypto_mode parameter relate to “mode based on the algorithm used to generate the AEK key”? Nohow, documentation and implementation are misleading. So, what is stored in aek.key file? Nothing but a sequence of bytes which represent either 3DES or AES secret key.

No about the problem.

I have not idea why, but in D7, when fallback settings take place, encrypttext API commands got an extremely weird behaviour:

Session id is s0
API> encrypttext,c,xxx
...
DM_ENCR_TEXT_V2=AAAACHYQW2Ab8FTGW8gul3tK6Q8M+9RKuRSGgxypEcVmqaJLt2JE0+7tuKzVQzh78QTCxS6gcWAq7sOx
API> encrypttext,c,xxx
...
DM_ENCR_TEXT=W8gul3tK6Q8M+9RKuRSGgzw1veDUWIeDnWK6wT7gFEaVCVEGGORiHu/uzJNPhsAh

moreover:

~]$ iapi -X -Sapi
Running with non-standard init level: api
API> encrypttext,a,xxx
...
DM_ENCR_TEXT_V2=AAAACHYQW2Ab8FTGW8gul3tK6Q8M+9RKuRSGg63EjkDYL8tY2+Ox0El+nLbK+UgeDk0lAKELAnUx2Rnu
API> encrypttext,a,xxx
...
DM_ENCR_TEXT=W8gul3tK6Q8M+9RKuRSGg5J4DiemgxHojpL7UY6GbwClg7osvnn1GTnEmC672QgY
API> 

What does such behaviour mean? Now, you are unable to set ldap password from DA: when setting ldap password from DA, it executes replicate_setup_methods docbase method (something like “execute do_method with method=’replicate_setup_methods’,arguments=’mkfile_encrypt_text password /u01/documentum/dba/config/DCMT_DEV/ldap_08002b8f801ccabb.cnt'”), this method executes encrypttext API command and gets wrong password.

2 thoughts on “Encryption madness. Part II

  1. Pingback: Encryption madness. Holy crap | Documentum in a (nuts)HELL
  2. Pingback: Encryption madness. Part IV | Documentum in a (nuts)HELL

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s