WTF???

I like large deployments because large deployments mean a lot of challenges 🙂 And the new challenge is: ACLs are fucking slow because EMC fails to write fast code. Let’s check how long does it take to create ACL with 60000 accessors on DFC side (I do know that it is not a good idea to create ACLs with more than 32767 accessors, but I like drawing cool charts):

public class DfAclTest extends DfcTestSupport {

    @Test
    public void aclTest() throws Exception {
        IDfSession session = getSession();
        int multiplier = 1000;
        IDfACL acl = (IDfACL) session.newObject("dm_acl");
        System.out.println("\n=========== CREATE ===========");
        long start = System.currentTimeMillis();
        for (int i = 1; i <= 60; i++) {
            grant(acl, (i - 1) * multiplier, multiplier);
            System.out.println("accessors: " + (i * multiplier) + ", time: "
                    + (System.currentTimeMillis() - start));

        }
        System.out.println("\n=========== UPDATE ===========");
        start = System.currentTimeMillis();
        for (int i = 1; i <= 60; i++) {
            grant(acl, (i - 1) * multiplier, multiplier);
            System.out.println("accessors: " + (i * multiplier) + ", time: "
                    + (System.currentTimeMillis() - start));

        }
    }

    private static void grant(IDfACL acl, int offset, int amount)
        throws DfException {
        for (int i = offset; i < amount + offset; i++) {
            acl.grant(String.valueOf(i), 3, null);
        }
    }

}

and results are really impressive:

=========== CREATE ===========
...
accessors: 60000, time: 1155926
...
=========== UPDATE ===========
...
accessors: 60000, time: 3564534

i.e. if we were need to create ACL with 60000 records it would take 20 minutes, if we decided to update each record in that ACL it would take one hour 🙂 I was unable to understand how it was possible to write so slow algorithm and decided to somehow optimize it, fortunately it didn’t take much time – I just changed two methods in com.documentum.fc.client.DfACL:

  1. findEntry – from
    for (int i = 0, limit = getAccessorCount(); i < limit; i++) {
        if (getAccessorPermitType(i) == permitType 
                && getAccessorName(i).equals(accessorName)) {

    to

            int start = findString(ACCESSOR_NAME, accessorName);
            if (start < 0) {
                return  -1;
            }
    
            for (int i = start, limit = getAccessorCount(); i < limit; i++) {
                if (getAccessorName(i).equals(accessorName)
                        && getAccessorPermitType(i) == permitType) {
    
  2. updateAuditTrail to
    return

and got another impressive results.

First optimization only:

=========== CREATE ===========
...
accessors: 60000, time: 456957
...
=========== UPDATE ===========
...
accessors: 60000, time: 537209

First and second optimizations:

=========== CREATE ===========
...
accessors: 60000, time: 136213
...
=========== UPDATE ===========
...
accessors: 60000, time: 131164

Chart:

3 thoughts on “WTF???

  1. Content Server and DFC version? We have also faced slow performance after the upgrade to 7.2 😦

    Like

  2. That is irrelevant to version, in case of ACLs EMC uses algorithms with polynomial complexity (i believe O(n^2)) and this is the problem. My version also suffers from polynomial complexity but coefficients are much better.

    As regards to your problem… Another guy also asked me about D7/Solaris performance, he had something like:

    "main" prio=3 tid=0x00000001004cc800 nid=0x1 runnable [0xffffffff7fffc000]
       java.lang.Thread.State: RUNNABLE
        at java.lang.Object.clone(Native Method)
        at com.rsa.jcm.f.dn.clone(Unknown Source)
        at com.rsa.jcm.f.hn.f(Unknown Source)
        at com.rsa.jcm.f.dn.k(Unknown Source)
        at com.rsa.jcm.f.cv.a(Unknown Source)
        at com.rsa.jcm.f.cv.m(Unknown Source)
        at com.rsa.jcm.f.cv.nextBytes(Unknown Source)
        - locked <0x00000007ec4d7588> (a com.rsa.jcm.f.cv)
        at com.rsa.jcm.f.cv.nextBytes(Unknown Source)
        at com.rsa.cryptoj.o.nt.engineNextBytes(Unknown Source)
        at java.security.SecureRandom.nextBytes(SecureRandom.java:455)
        - locked <0x00000007ec4d6fc8> (a java.security.SecureRandom)
    

    and his problem was described here: https://blog.documentum.pro/2016/04/11/encryption-madness-part-iii/
    I would suggest you to make a couple of thread dumps and send them to me for analysis.

    Like

  3. Pingback: ACL performance | Documentum in a (nuts)HELL

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s