DFS challenge

Yesterday I read interesting opinion about DFS on ECN:

I would strongly advise against using DFS, you should check if the REST API fits your use case: DFS is built on outdated libraries and it is “difficult” to make it work in certain (most) environments and with any java version newer than 6.0

Unfortunately, it is only a half of truth – at current moment there are no reliable options to exchange data between Documentum and other systems, and the fact that DFS is unreliable does not make REST API robust. So, what is wrong with DFS? I believe “DFS is built on outdated libraries” means following: DFS client libraries depend on ancient version of jaxws-rt, i.e. 2.1.7, and when we try to run DFS client against recent version of jaxws-rt we get something like:

java.lang.NoSuchMethodError: com.sun.xml.ws.api.message.Message.getHeaders()Lcom/sun/xml/ws/api/message/HeaderList;
at com.emc.documentum.fs.rt.impl.pipe.DfsTube.processRequest(DfsTube.java:89)
at com.sun.xml.ws.api.pipe.Fiber.__doRun(Fiber.java:1136)
at com.sun.xml.ws.api.pipe.Fiber._doRun(Fiber.java:1050)
at com.sun.xml.ws.api.pipe.Fiber.doRun(Fiber.java:1019)
at com.sun.xml.ws.api.pipe.Fiber.runSync(Fiber.java:877)
at com.sun.xml.ws.client.Stub.process(Stub.java:463)
at com.sun.xml.ws.client.sei.SEIStub.doProcess(SEIStub.java:191)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:108)
at com.sun.xml.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:92)
at com.sun.xml.ws.client.sei.SEIStub.invoke(SEIStub.java:161)
at com.sun.proxy.$Proxy41.get(Unknown Source)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at com.emc.documentum.fs.rt.context.impl.compat.ClientReflectionServiceInvokerCompat60.invoke(ClientReflectionServiceInvokerCompat60.java:56)
at com.emc.documentum.fs.rt.context.impl.UcfClientInvocationHandler.invoke(UcfClientInvocationHandler.java:52)
at com.emc.documentum.fs.rt.context.impl.SoapClientInvocationHandler.invoke(SoapClientInvocationHandler.java:60)
at com.emc.documentum.fs.rt.context.impl.MtomCompatHandler60SP1.invoke(MtomCompatHandler60SP1.java:60)
at com.emc.documentum.fs.rt.context.impl.HttpSessionInvocationHandler.invoke(HttpSessionInvocationHandler.java:88)
at com.emc.documentum.fs.rt.context.impl.RemoteServiceInterfaceInvocationHandler.invoke(RemoteServiceInterfaceInvocationHandler.java:30)
at com.emc.documentum.fs.rt.context.impl.ReturnedContentTransformationHandler.invoke(ReturnedContentTransformationHandler.java:45)
at com.emc.documentum.fs.rt.context.impl.OperationOptionsHandler.invoke(OperationOptionsHandler.java:74)
at com.emc.documentum.fs.rt.context.impl.ContextThreadLocalInvocationHandler.invoke(ContextThreadLocalInvocationHandler.java:48)
at com.emc.documentum.fs.rt.context.impl.ServiceContextInvocationHandler.invoke(ServiceContextInvocationHandler.java:30)
at com.emc.documentum.fs.rt.context.impl.FileRegistryCleanupHandler.invoke(FileRegistryCleanupHandler.java:24)
at com.sun.proxy.$Proxy25.get(Unknown Source)

which could be fixed via following code (technically it just recompiles the problem com.emc.documentum.fs.rt.impl.pipe.DfsTube#processRequest method agains recent version of jaxws-rt):

/**
 * @author Andrey B. Panfilov <andrey@panfilov.tel>
 */
public class FixDfsTube {

    public static void main(String[] args) throws Exception {
        ClassPool pool = ClassPool.getDefault();
        for (Class cls : new Class[] {ServiceContextAdapter.class,
            ThreadLocalContextStorage.class, Header.class, StringUtils.class,
            DOMHeader.class, ContentTransferModeUtil.class }) {
            pool.importPackage(cls.getPackage().getName());
        }

        String cls = DfsTube.class.getName();
        CtClass cc = pool.get(cls);
        CtMethod m = cc.getDeclaredMethod("processRequest");
        StringBuilder body = new StringBuilder();
        body.append("{\n");
        body.append("ServiceContextAdapter contextAdapter = (ServiceContextAdapter) ThreadLocalContextStorage\n");
        body.append("        .get();\n");
        body.append("String serviceName = $1.endpointAddress.getURL().getFile();\n");
        body.append("if (serviceName.endsWith(CONTEXT_REGISTRY_SERVICE_NAME)\n");
        body.append("        || serviceName.endsWith(AGENT_SERVICE_SERVICE_NAME)\n");
        body.append("        || contextAdapter == null) {\n");
        body.append("    return super.processRequest($1);\n");
        body.append("}\n");
        body.append("Header header;\n");
        body.append("if (StringUtils.isNotBlank(contextAdapter.getConsolidatedContext()\n");
        body.append("        .getToken())) {\n");
        body.append("    header = new DOMHeader(addTokenToHeader(contextAdapter\n");
        body.append("            .getConsolidatedContext().getToken()));\n");
        body.append("    $1.getMessage().getHeaders().add(header);\n");
        body.append("}\n");
        body.append("\n");
        body.append("if (!contextAdapter.isDeltaEmpty()) {\n");
        body.append("    header = new DOMHeader(\n");
        body.append("            createDeltaContextElement(contextAdapter.getDeltaContext()));\n");
        body.append("    $1.getMessage().getHeaders().add(header);\n");
        body.append("}\n");
        body.append("if (ContentTransferModeUtil.isMtomTransfer(contextAdapter)) {\n");
        body.append("    enableMtom($1);\n");
        body.append("} else {\n");
        body.append("    disableMtom($1);\n");
        body.append("}\n");
        body.append("return super.processRequest($1);\n");
        body.append("}");
        m.setBody(body.toString());

        String resourceName = DfsTube.class.getName().replace(".", "/")
                + ".class";
        URL resourceURL = DfsTube.class.getResource("/" + resourceName);
        JarURLConnection connection = (JarURLConnection) resourceURL
                .openConnection();
        URL jarURL = connection.getJarFileURL();
        String fileName = jarURL.getFile();
        ZipFile zipFile = new ZipFile(fileName);
        ZipOutputStream zos = new ZipOutputStream(new FileOutputStream(fileName
                + ".patched"));
        for (Enumeration e = zipFile.entries(); e.hasMoreElements();) {
            ZipEntry entryIn = (ZipEntry) e.nextElement();
            if (!entryIn.getName().equals(resourceName)) {
                zos.putNextEntry(new ZipEntry(entryIn.getName()));
                InputStream is = zipFile.getInputStream(entryIn);
                byte[] buf = new byte[1024];
                int len;
                while ((len = is.read(buf)) > 0) {
                    zos.write(buf, 0, len);
                }
            } else {
                zos.putNextEntry(new ZipEntry(resourceName));
                zos.write(cc.toBytecode());
            }
            zos.closeEntry();
        }
        zos.close();
    }

}

Enjoy 🙂

6 thoughts on “DFS challenge

  1. It’s not just that, that is just one of the problems. Try to develop a spring-based client and add DFS client libraries: You’ll end up with a nice mix of librearies expecting a Java >=7, others Java 6 and such and every one of them expecting different signatures from the same methods (which, can be overriden with your workaround)

    Then try to deploy it on WAS, (and, if possible, avoid the “suggested” method by documentation to make it work: replace jars in WAS lib folder, affecting every other application deployed).

    Rest API is not more robust, but at least you don’t have to fight with outdated library versions. IMHO, it’s not worth the time and “workarounds” required to make it work, when you have an “easier” alternative such as rest services (which at least it looks like it’s been “actively” updated, instead of the support note saying “customers asked for a Java7 updated DFS but it is not on the roadmap”)

    Like

  2. The short story: I ready to accept a challenge – share your code and I will demonstrate how to run it in J2EE environment, what I have demonstrated does work on JRE8.

    The long story: REST API is better because there is no OOTB client libraries (no client libraries – no outdated versions), ECD tried to create something, but their code is a piece of dog crap: https://blog.documentum.pro/2016/04/28/keep-your-pathos-to-yourself/, from the DFS perspective EMC did a fatal mistake – they perform authentication using custom SOAP headers, which cause a lot of pain in behind if you want to create your own client from scratch, on the other hand XML/WS API in Java was always a pain in ass, but at current moment webservices is the only reliable option to build integration.

    PS. That is how REST works since 3 (or 4) years or initial release: https://community.emc.com/docs/DOC-55482#comment-46772

    Like

  3. I never really liked the DFS as it was always way too much overhead of marshalling and unmarshalling the payload data. Many system “integrations” (where Dctm was used as an archive) I had seen (a while back) consistent of “give me list of related documents” and “show me that one”. And for that having to use a boat load of libraries for e.g. those two simple use cases seemed like overkill to me. I was working in a highly controlled change management using Maven to deploy to WebSphere. To summarize, the less libraries you have to manage the better… (version conflicts as described above, for starters).

    With the early developer release of the rest services I saw how a (very simple) repository browser was possible using html + some javascript made me never look back.

    Like

  4. I believe everybody does know my opinion about EMC/ECD/IGG (and soon I will publish my opinion about OT:) ), so, I’m not going to advocate DFS – it is clear that implementation is unreliable, but your arguments seem to be wrong, let’s elaborate:

    I was working in a highly controlled change management using Maven to deploy to WebSphere. To summarize, the less libraries you have to manage the better…

    I do not see any problem there, there is pom.xml for emc-dfs-services-remote: https://github.com/andreybpanfilov/prodctm/blob/master/miscellaneous/dfs-install/src/main/resources/emc-dfs-services-remote.xml, below is maven dependency for my maven project:

            <dependency>
                <groupId>com.emc.documentum.dfs</groupId>
                <artifactId>emc-dfs-services-remote</artifactId>
                <version>${dfs.version}</version>
            </dependency>
    

    Where is a jar hell?

    With the early developer release of the rest services I saw how a (very simple) repository browser was possible using html + some javascript made me never look back.

    If you believe that ECD’s REST API is simple enough try do discover scaffolding concept: https://www.youtube.com/watch?v=lkZxRSS0Zwg

    Like

  5. I’ve never said it “won’t” work 😀 I’ve said it’s not worth the effor. We’ve made a working version (completely unsupported of course :D), but we almost spend more time fixing incompatibilities between libraries (note: everything worked fine in tomcat 6/7 + java 6/7/8, but requirements changed to deploy on WAS 8 + Java 7, and that was hell) than the time we spend developing the client.

    As you said, the main advantage of REST is the lack of a hundred different libraries, not the “quality” of the code, but I still prefer to avoid wasting time trying to make a dfs client work as it should, when (and if) I can do the same with REST, just because EMC was lazy and didn’t want to update the DFS.

    PS. People I know that work with OpenText said their products are “bad” and that support is even worst, some of them worked beofre with Documentum and said that we’ll most likely will miss EMC support… o_O

    Like

  6. Pingback: Why you should stay clear of REST | Pro Documentum

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s