Think soberly. Maturity myth

Original could be found there and there:

Documentum in Maintenance Mode

One thing that was apparent throughout the event was Documentum and existing interfaces (Webtop, D2, xCP) moving into more of a maintenance mode. Some examples included:

  • Rohit’s quote that Documentum was “feature complete” during the keynote.
  • Subsequent article in CMSWire that “No one is asking for more features or functions”
  • Documentum Roadmap that focused on releases that will make Documentum easier to upgrade.

Other expectations – moving beyond

Documentum is feature rich – not clamoring for more capability. As we pointed out from the morning Documentum 7 session, ECD is treating Documentum as complete and has focused on making it simpler to run Documentum.

Actually, the statement about “feature complete” is totally weird. If my memory serves me right, today is 2016 but Documentum still suffers from childhood diseases like:

  • Partial support of unicode
  • Extremely limited SOAP/REST API (please check my thoughts on ECN, actually, in my last project, I’m trying to map CRUD operations on DFC calls and got some progress on that)
  • Poor integration capabilities
  • Java-only API
  • non J2EE compliant DFC library
  • Poor java-api
  • Poor documentation
  • Poor performance
  • Poor security
  • Poor support

This is how “mature application” does look from EMC perspective. Actually, my opinion about maturity of Documentum is following:

  • in 2007 EMC acquired XHive and after that they implemented XHive storage support in Documentum (I have never used it)
  • further EMC implemented xPlore (also based on XHive), technically xPlore have no search capabilities at all: it grabs all data from database and storage, puts it into another big database and tries to performs XML queries against xml database
  • further EMC implemented xCP2 which relies on xMS which in turn based on XHive
  • further EMC implemented InfoArchive that is based on XHive
  • now EMC is trying to implement Horizon/LEAP which is also based on XHive
  • the co-founder of XHive was Jeroen van Rotterdam, who is ECD CTO in EMC now

From my perspective it is obvious that ECD/IGG/etc development vector had switched to XHive long time ago.

Think soberly. Is it hard?

Well, Momentum ended, TSGRP guys posted theirs thoughts about conference, now it is a time to ruin the myths.

It is worth to say that I have never attended such eventsMomentum (and most probably will never do, no matter what company will held it in the future), because from my perspective such events should look like:

EMC thinks that Momentum looks like:

but in real life it looks like:

Docker myth

Issue #1 to address – Forced re-indexing and database changes are time consuming

Patrick offered a stateless Documentum –

  • Point and run Content Server
  • Binary upgrades
  • No forced changes to database
  • Eliminates downtime from reindexing

Able to bring in Bedrock server without any changes to database or repository.

Makes upgrades easier (and faster). Has the ability to just run “Bedrock” against existing database and content repository due to everything due to Documentum in a Docker Container.

  • Portable Production Ready Images
  • Continuous delivery of upgrades and patches to production
  • Faster Rollback

Documentum Application Containers

  • Developer – All in One container for Functional Investigation
  • Service Images – Individual services containers for Agile Expansion and Load balancing
  • Stateless Documentum – External Content, Database & Isolated Configuration for Fast Patching and Upgrades

Different Containers for Content Store could be more containers for xCP or other customizations.
Docker is Linux based (no support for Microsoft right now).

How to migrate to Docker environment.

  • Add content server – Docker container – can operate alongside existing Documentum
  • Deploy extra Docker containers to manage high volumes
  • Migrate to Docker by turning off original non-Docker instances

How many people are looking at Docker (only 1 out of 60 in room)
Hot upgrades to D7.3+ (Bedrock – Feldspar…)

  • Can run Bedrock on Docbase 7.2 and also have it work on a Docbase Bedrock.
  • Run new version along side old version in Production

Actually, everything that was “written” about Docker sounds ridiculous… The main problem is: Docker complicates everything. Why? Let’s elaborate. First of all, what is a Docker image? It is just a tar archive where maintainer put service’s binaries among with dependencies and bootstrap procedures: take a look at my shell scenario which creates document image – there is nothing related to Docker except the command which uploads archive into the Docker repository, there is no rocket science: I may unpack my archive into specific directory and run my services inside chroot take a look at chroot usecases – they are pretty the same as Docker’s:

  • Testing and development
  • Dependency control
  • Compatibility
  • Recovery
  • Privilege separation

If chroot is so cool, what are the Docker’s advantages over chroot? I do believe that the answer for this question is obvious:

But if you have chosen to use Docker you must understand that your deployment/maintenance workflow changes dramatically, for example, imagine that you decided to install a patch/update in Dockerized environment (no matter Documentum (i.e. application) or operating system (i.e. platform) patch), what steps do you need to perform to achieve your goal? These steps are:

  • update base image
  • test new base image
  • stop existing container
  • remove existing container
  • deploy new container

This procedure, obviously, is less straightforward than installing patches/updates using puppet, ansible or something else or even manually, however, this is how Docker workflow designed and supposed to work. Now ask yourself how many typical Documentum installation do you need to have to start taking advantage of using Docker? In my opinion dozen is not enough, if you have more you are either a large enterprise or a big consulting company, and it is not a EMC business to lecture you how to manage your environments.

Persistence API. Part II

I got a couple of comments for previous blogpost, and some guys had contacted me via email. Unfortunately, the all thoughts was “yeah, I would like to have persistence API for Documentum, please share it”, but nobody had written something like “I would like to participate in such project” 😦 On the other hand I have expected that, so, no worries 🙂

Well, I have shared my project on github, if you want to participate – you are welcome, if you expect to get something usable during next two-three months – it is not possible, because I can’t spend all my time on this project.

UPD

There is another similar project on github: xcp-persistence

What DFC version you are on?

Two months ago EMC announced availability of D2 4.6, by tradition there are a lot of “security enhancements” and bug fixes (though I’m not going to describe all security enhancements right now, but it worth to note, the EMC employees failed to answer whether it is safe to use D2 or not from security perspective). Interesting fact that EMC finally released D2 REST API, though they promised to do that at the end of year (not sure what calendar they use, in case of Chinese calendar everything is OK). Interesting thing here is a contents of D2 REST distribution:

$ a=D2-REST-4.6.0.war;zipinfo -1 $a "*.jar"|sed 's!-[0-9].*!*!'|uniq -d|xargs zipinfo $a
  168760 b- defN  5-Jan-16 05:06 WEB-INF/lib/commons-beanutils-core-1.7.0.jar
  206035 b- defN  9-Dec-14 07:33 WEB-INF/lib/commons-beanutils-core-1.8.0.jar
   46725 b- defN  9-Dec-14 08:00 WEB-INF/lib/commons-codec-1.3.jar
   58160 b- defN  5-Jan-16 05:01 WEB-INF/lib/commons-codec-1.4.jar
  575389 b- defN  9-Dec-14 07:33 WEB-INF/lib/commons-collections-3.2.1.jar
  571259 b- defN  5-Jan-16 05:02 WEB-INF/lib/commons-collections-3.2.jar
  271849 b- defN  5-Jan-16 05:06 WEB-INF/lib/commons-configuration-1.5.jar
  298829 b- defN  9-Dec-14 07:33 WEB-INF/lib/commons-configuration-1.6.jar
   59590 b- defN  5-Jan-16 05:04 WEB-INF/lib/commons-fileupload-1.2.2.jar
   68622 b- defN  9-Dec-14 08:03 WEB-INF/lib/commons-fileupload-1.3.jar
  109043 b- defN  5-Jan-16 05:01 WEB-INF/lib/commons-io-1.4.jar
  159509 b- defN  9-Dec-14 08:03 WEB-INF/lib/commons-io-2.0.1.jar
  261809 b- defN  5-Jan-16 05:01 WEB-INF/lib/commons-lang-2.4.jar
  279193 b- defN  9-Dec-14 08:01 WEB-INF/lib/commons-lang-2.5.jar
    9207 b- defN  9-Jan-15 05:02 WEB-INF/lib/configservice-api-7.2.0-20150109.120223-17.jar
    9207 b- defN 19-Feb-16 17:08 WEB-INF/lib/configservice-api-7.2.0-SNAPSHOT.jar
  115166 b- defN  9-Jan-15 05:02 WEB-INF/lib/configservice-impl-7.2.0-20150109.120332-17.jar
  115168 b- defN 19-Feb-16 17:08 WEB-INF/lib/configservice-impl-7.2.0-SNAPSHOT.jar
15301367 b- defN  9-Jan-15 05:02 WEB-INF/lib/dfc-7.2.0-20150109.120126-17.jar
15569763 b- defN 19-Feb-16 17:08 WEB-INF/lib/dfc-7.2.0-SNAPSHOT.jar
    6139 b- defN 10-Jan-15 05:02 WEB-INF/lib/dms-client-api-7.2.0-20150109.141907-6.jar
    6138 b- defN  5-Jan-16 05:09 WEB-INF/lib/dms-client-api-7.2.0-SNAPSHOT.jar
  228286 b- defN  5-Jan-16 05:05 WEB-INF/lib/jackson-core-asl-1.9.2.jar
  228315 b- defN  9-Dec-14 07:32 WEB-INF/lib/jackson-core-asl-1.9.4.jar
  765648 b- defN  5-Jan-16 05:05 WEB-INF/lib/jackson-mapper-asl-1.9.2.jar
  777693 b- defN  9-Dec-14 07:32 WEB-INF/lib/jackson-mapper-asl-1.9.4.jar
  481535 b- defN  5-Jan-16 05:02 WEB-INF/lib/log4j-1.2.16.jar
  489884 b- defN  9-Dec-14 08:01 WEB-INF/lib/log4j-1.2.17.jar
   23659 b- defN  2-Mar-16 14:14 WEB-INF/lib/slf4j-api-1.5.10.jar
   25496 b- defN  9-Dec-14 08:03 WEB-INF/lib/slf4j-api-1.6.1.jar
    9648 b- defN  5-Jan-16 05:11 WEB-INF/lib/slf4j-log4j12-1.5.5.jar
    9753 b- defN  9-Dec-14 08:03 WEB-INF/lib/slf4j-log4j12-1.6.1.jar

Yeah, even building a war-archive is already a challenge 😦

Persistence API

When year ago EMC announced the work under Persistence API for Documentum I started to worry about the future of their product stack (D2, xCP, etc) – who the hell needs xCP if you can create your own domain model and take advantage of modern frameworks like wicket, vaadin, etc? Fortunately, my concerns were not justified – EMC shared their work on github, and if even forget about the fact they they did nothing during last year:

I can definitely say that their implementation is a piece of shit. Interesting, what does EMC think about transactions? Nothing:

What does EMC think about following their recommendations? Nothing:

By the way I have started to work under JDO implementation for DFC, where code looks like:

<?xml version="1.0" encoding="UTF-8" ?>
<persistence xmlns="http://java.sun.com/xml/ns/persistence"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://java.sun.com/xml/ns/persistence
        http://java.sun.com/xml/ns/persistence/persistence_2_0.xsd" version="2.0">

    <persistence-unit name="Testing">
        <class>pro.documentum.model.DmUser</class>
        <exclude-unlisted-classes/>
        <properties>
            <property name="javax.jdo.option.ConnectionURL" value="dctm:DCTM_DEV"/>
            <property name="javax.jdo.option.ConnectionUserName" value="dmadmin"/>
            <property name="javax.jdo.option.ConnectionPassword" value="dmadmin"/>
            <property name="datanucleus.identityStringTranslatorType" value="dfidtranslator"/>
        </properties>
    </persistence-unit>
</persistence>
import java.util.Date;

import javax.jdo.annotations.Column;
import javax.jdo.annotations.PersistenceCapable;

import lombok.Getter;
import lombok.Setter;

/**
 * @author Andrey B. Panfilov <andrey@panfilov.tel>
 */
@PersistenceCapable(table = "dm_user")
public class DmUser extends AbstractPersistentObject {

	@Column(name = "user_name")
	@Getter
	@Setter
	private String userName;

	@Column(name = "user_os_name")
	@Getter
	@Setter
	private String userOSName;

	@Column(name = "user_address")
	@Getter
	@Setter
	private String userAddress;

	@Column(name = "user_group_name")
	@Getter
	@Setter
	private String userGroupName;
}


/**
 * @author Andrey B. Panfilov <andrey@panfilov.tel>
 */
public class TestJDO {

	@Test
	public void testName() throws Exception {
		PersistenceManagerFactory pmf = JDOHelper.getPersistenceManagerFactory("Testing");
		PersistenceManager pm = pmf.getPersistenceManager();
		Query query = pm.newQuery(DmUser.class, "(userName == :user_name || userLoginName == :user_name)");
		List<DmUser> results = (List<DmUser>) query.executeWithArray("dmadmin");
		assertNotNull(results);
		assertEquals(1, results.size());
		DmUser user = results.get(0);
		assertEquals("dmadmin", user.getUserName());
		assertEquals(16, user.getUserPrivileges());
        pm.close();
	}

}

and if you are interested in existence of robust Persistence API for Documentum let me know – I will share my work.