Session management. Horse races

Just raw results without explanation…

Horses: Session, SessionI, SessionII, SessionIII, SessionIV, SessionV, SessionVI, SessionVII, DarkHorse – implement different patterns to acquire DFC-session

DFC Settings:

  • default – means default settings
  • reuse_limit – dfc.session.reuse_limit = 2147483647
  • global_pool – dfc.session.global_pool_enabled=true
  • old pool – dfc.compatibility.useD7SessionPooling=false

Results: DFC benchmark

Preview:



DNF (did not finish) means enabling global pool causes thread-safety issues like:

java.util.NoSuchElementException
        at java.util.LinkedList.getFirst(LinkedList.java:109)
        at com.documentum.fc.client.impl.session.GlobalSessionPool.get(GlobalSessionPool.java:41)
        at com.documentum.fc.client.impl.session.PooledSessionFactory.newSession(PooledSessionFactory.java:33)
        at com.documentum.fc.client.impl.session.SessionManager.getSessionFromFactory(SessionManager.java:134)
        at com.documentum.fc.client.impl.session.SessionManager.newSession(SessionManager.java:72)
        at com.documentum.fc.client.impl.session.SessionManager.getSession(SessionManager.java:191)
        at tel.panfilov.documentum.benchmark.impl.Session.doOp(Session.java:31)
        at tel.panfilov.documentum.benchmark.impl.SessionI.doOp(SessionI.java:14)
        at tel.panfilov.documentum.benchmark.Benchmark.run(Benchmark.java:105)
        at java.lang.Thread.run(Thread.java:662)


java.util.ConcurrentModificationException
        at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:761)
        at java.util.LinkedList$ListItr.remove(LinkedList.java:729)
        at com.documentum.fc.client.impl.session.GlobalSessionPool.flush(GlobalSessionPool.java:114)
        at com.documentum.fc.client.impl.session.PooledSessionFactory.flush(PooledSessionFactory.java:80)
        at com.documentum.fc.client.impl.session.SessionManager.flushSessions(SessionManager.java:259)
        at com.documentum.fc.client.impl.session.SessionManager.flushSessions(SessionManager.java:287)
        at tel.panfilov.documentum.benchmark.impl.SessionIII.doOp(SessionIII.java:15)
        at tel.panfilov.documentum.benchmark.Benchmark.run(Benchmark.java:105)
        at java.lang.Thread.run(Thread.java:662


Exception in thread "Global Session pool worker" java.util.ConcurrentModificationException
        at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:761)
        at java.util.LinkedList$ListItr.next(LinkedList.java:696)
        at com.documentum.fc.client.impl.session.GlobalSessionPool.flushExpiredSessions(GlobalSessionPool.java:203)
        at com.documentum.fc.client.impl.session.GlobalSessionPool$ExpirationThread.run(GlobalSessionPool.java:232)

Time in Documentum

Since D6 release EMC changed the manner of storing dates in database – now CS stores dates in UTC by default, the problem is the new settings are totally undocumented.

Misleading documentation

Powerlink states:

  1. The r_normal_tz property, in the docbase config object controls how Content Server stores dates in the repository. If set to 0, all dates are stored in UTC time. If set to an offset value, dates are normalized using the offset value before being stored in the repository. If set to an offset value, the property must be set to a time zone offset from UTC time, expressed as seconds. For example, if the offset represents the Pacific Standard Time zone, the offset value is -8*60*60, or -28800 seconds. When the property is set to an offset value, Content Server stores all date values based on the time identified by the time zone offset.
    Refer to the Content Server Administration Guide V6.0 for more information about how the value set for this attribute is used to set the timestamp, depending on whether the client is 6.0 and up or pre-6.0.
  2. To answer the question on how this value is set:
    In a new Documentum 6 or later repository, r_normal_tz is set to 0. In a repository upgraded from a release prior to Version 6, r_normal_tz is set to the offset representing Content Server local time. Therefore, if set and this value was not set manually, this was probably an upgrade from a pre-6.0 version Docbase.
  3. The r_tz_aware set to FALSE makes the Content Server not aware of the time zone.
    This attribute is not documented presently (I do not know why), but if the customer’s r_normal_tz is set to a non-zero value, then they probably upgraded their docbase and possibly FALSE is the default value for this attribute in the case of an upgrade.

This KB article is absolutely incorrect, lets explain.

r_normal_tz

At first glance it is a totally stupid idea to normalize dates using static offset, how are they going to manage daylight saving time? Change r_normal_tz every half of year and restart server? Lets check what happens if we change r_normal_tz in docbase config:

Default settings (r_normal_tz=0, r_tz_aware=T):

Connected to Documentum Server running Release 7.0.0100.0603  Linux.Oracle  
1> select r_normal_tz, r_tz_aware, r_creation_date from dm_docbase_config  
2> go  
r_normal_tz   r_tz_aware    r_creation_date  
------------  ------------  -------------------------  
           0             1  10/31/2013 13:29:41  
(1 row affected)
SQL> ALTER SESSION set NLS_DATE_FORMAT='YYYY-MM-DD HH24:MI:SS';  
  
Session altered.  
  
SQL> SELECT r_creation_date AS r_creation_date_utc,  
       CAST (  
          (FROM_TZ (CAST (r_creation_date AS TIMESTAMP), '+00:00')  
              AT TIME ZONE 'Europe/Moscow') AS DATE)  
          AS r_creation_date  
  FROM dm_docbase_config_sp;  
  
R_CREATION_DATE_UTC R_CREATION_DATE  
------------------- -------------------  
2013-10-31 09:29:41 2013-10-31 13:29:41 

UTC+10 offset:

Connected to Documentum Server running Release 7.0.0100.0603  Linux.Oracle  
1> select r_normal_tz, r_tz_aware, r_creation_date from dm_docbase_config  
2> go  
r_normal_tz   r_tz_aware    r_creation_date  
------------  ------------  -------------------------  
       36000             1  10/31/2013 09:29:41  
(1 row affected) 

UTC-10 offset:

Connected to Documentum Server running Release 7.0.0100.0603  Linux.Oracle  
1> select r_normal_tz, r_tz_aware, r_creation_date from dm_docbase_config  
2> go  
r_normal_tz   r_tz_aware    r_creation_date  
------------  ------------  -------------------------  
      -36000             1  10/31/2013 09:29:41  
(1 row affected)  

So, value of r_normal_tz parameter has nothing in common with timezone: if r_normal_tz=0 CS converts dates to UTC before storing in database, if r_normal_tz!=0 CS stores dates without conversion, e.g. database dates are local. Actually this new behavior has some issues:

  1. When you use some reporting software and write reports against database you should take into account this “feature” and cast dates to local time
  2. UTC timescale is always straightforward but localtime not due to DST. So, there are some ambiguities in converting localtime to UTC and backward, i.e. 2010-10-30 22:30:00 UTC and 2010-10-30 23:30:00 UTC dates have the same representation for Moscow tmezone, so, if you were “lucky” and created document between 2010-10-30 22:30:00 UTC and 2010-10-30 23:30:00 UTC you can’t find it by creation date, because DFC will convert dates you input in dates after 2010-10-30 23:30:00 UTC

r_tz_aware

My dctmpy library is able to emulate two major versions of DFC (e.g. pre-D6 and post-D6) and allows to view traffic passed between CS and client easier than capture network traffic or parse dfc logs. Test script:

#!python  
from dctmpy.docbase import Docbase  
  
  
def main():  
    session = Docbase(host="192.168.2.56", port=12000)  
    session.authenticate("dmadmin", "dmadmin")  
    for e in session.query("SELECT r_creation_date as dt FROM dm_docbase_config"):  
        print e.__buffer__  
  
  
if __name__ == "__main__":  
    main() 

little patch to show serialized data:

Index: dctmpy/obj/collection.py  
===================================================================  
--- dctmpy/obj/collection.py    (revision 34)  
+++ dctmpy/obj/collection.py    (working copy)  
@@ -110,7 +110,9 @@  
  
class CollectionEntry(TypedObject):  
     def __init__(self, **kwargs):  
+        b = kwargs.get("buffer")  
         super(CollectionEntry, self).__init__(**kwargs)  
+        self.__buffer__ = b[0: len(b) - len(self.buffer)]  
  
     def readHeader(self):  
         pass 

post-D6 traffic (note that now my server thinks that all previous dates are local though they was stored initially in UTC):

OBJ QR 0 0 0 1  
B S 4 2013-10-31T05:29:41Z  
0  
0  

pre-D6 traffic:

OBJ QR 1  
xxx Oct 31 09:29:41 2013  
0

So, D6 clients use ISO 8601 format to transfer dates while old clients use some “proprietary” protocol. Now what will happen if we switch r_tz_aware to false?

DFC:

1> select r_normal_tz, r_tz_aware, r_creation_date from dm_docbase_config  
2> go  
r_normal_tz   r_tz_aware    r_creation_date  
------------  ------------  -------------------------  
      -36000             0  10/31/2013 09:29:41  
(1 row affected)  

python:

OBJ QR 0 0 0 1  
B S 4 xxx Oct 31 09:29:41 2013  
0  
0

So, setting r_tz_aware to false switches “date protocol” to pre-D6 version. In practice this means that if you have DFC-clients, that use timezone different from CS, those clients will send and receive wrong data:

~]$ cat > Test.java  
import com.documentum.com.DfClientX;  
import com.documentum.fc.client.IDfSession;  
import com.documentum.fc.common.DfException;  
import com.documentum.fc.common.DfLoginInfo;  
  
/** 
* @author Andrey B. Panfilov <andrew@panfilov.tel> 
*/  
public class Test {  
  
    public static void main(String[] argv) throws DfException {  
        IDfSession session = new DfClientX().getLocalClient().newSession(  
                "ssc_dev", new DfLoginInfo("dmadmin", "dmadmin"));  
        System.out.println(session.getDocbaseConfig()  
                .getTime("r_creation_date")  
                .asString("yyyy.MM.dd G 'at' HH:mm:ss z"));  
        session.disconnect();  
    }  
  
}  
~]$ javac Test.java  
~]$ java Test  
2013.10.31 н.э. at 09:29:41 MSK  
~]$ java -Duser.timezone=Asia/Vladivostok Test  
2013.10.31 н.э. at 09:29:41 VLAT  

DFC

DFC-clients as all java-based clients use their own calendar instead of operating system calendar, also EMC introduced a dfc.time_zone parameter:

# The timezone of this DFC instance.
#
# This value is initialized from the Java Virtual Machine at startup time and
# normally doesn’t need to be specified. Legal values are the timezone IDs
# supported by the Java Virtual Machine.
#
dfc.time_zone =

Now we know that CS and DFC interact with each other using ISO 8601 date format, so what is the purpose of dfc.time_zone parameter? It just helps to initialize instances of SimpleDateFormat, used internally by DFC, with predefined timezone but does not change dates. It’s useful when you want to display dates casted to specific timezone but not able to setup that timezone for current environment (UNIXes has TZ environment variable, Windows – not) or set -Duser.timezone property for java-based application (like IDQL or IAPI):

Connected to Documentum Server running Release 7.0.0100.0603  Linux.Oracle  
Session id is s0  
API> get,c,apiconfig,dfc.time_zone  
...  
Europe/Moscow  
API> get,c,docbaseconfig,r_creation_date  
...  
10/31/2013 09:29:41  
API> set,c,apiconfig,dfc.time_zone  
SET> Asia/Vladivostok  
...  
OK  
API> connect,ssc_dev,dmadmin,dmadmin  
...  
s1  
API> get,c,docbaseconfig,r_creation_date  
...  
10/31/2013 16:29:41

Switching to r_normal_tz=0 from r_normal_tz!=0

Following SQL scenario will help you to generate SQL updates for all date fields:

SET LINES 300
SET PAGES 0
SET TRIMSPOOL ON

  SELECT    CASE
           WHEN ROW_NUMBER ()
                OVER (PARTITION BY utc.table_name ORDER BY utc.column_name) =
                   1
           THEN
              'UPDATE ' || utc.TABLE_NAME || ' SET '
        END
     || utc.COLUMN_NAME
     || ' = DECODE('
     || utc.COLUMN_NAME
     || ', NULL, NULL'
     || ', TO_DATE(''0001/01/01'', ''YYYY/MM/DD''), TO_DATE(''0001/01/01'', ''YYYY/MM/DD'')'
     || ', CAST ((FROM_TZ (CAST ('
     || utc.COLUMN_NAME
     || ' AS TIMESTAMP), ''Europe/Moscow'') AT TIME ZONE ''UTC'') AS DATE))'
     || CASE
           WHEN ROW_NUMBER ()
                OVER (PARTITION BY utc.table_name
                      ORDER BY utc.column_name DESC) <> 1
           THEN
              ','
           ELSE
              ';' || CHR (10) || 'COMMIT;'
        END
FROM user_tab_columns utc, user_tables ut
   WHERE utc.data_type = 'DATE' AND utc.table_name = ut.table_name
ORDER BY utc.table_name, utc.column_name;

How to provide backward compatibility with old java clients during UCF updates

This post was originally published on ECN.

If you are not familiar with apache httpd’s mod_rewrite and regexps, please do not read this post.

Problem

When EMC releases new UCF fixes, aimed to provide compatibility with corresponding JRE security fixes, they completely miss the fact, that customers are unable to perform JRE updates for all business users at the same time, and, so, IT personal should choose the lesser of two evils: either some users should suffer or all users should stay with vulnerable JRE, moreover, sometimes I can’t understand what EMC is doing (see also: Dumb UCF applet, Dumb UCF applet. Part II, UCF applet’s certificate expired… ORLY?)

Solution

At first we need to understand what files EMC changes from one point fix to another (it’s very simple for me because I download every new pointfix to check that EMC still not fixed 30 security issues ). Typically they change wdk/system/ucfinit.jar and wdk/fileselector/fileSelector.jar files, but ucfinit.jar contains checksums for some files in wdk/contentXfer directory, so, ucfinit.jar file in general is not interchangeable between point fixes. Major changes in webtop 6.7SP2 was (actually I double checked 1.7_45 and 1.7_25 versions I can certainly say that patch notes lie: for both versions it’s required to relax security settings to get working UCF):

  • P11 – JRE1.7_51 support
  • P07 – JRE1.7_45 support
  • P05 – JRE1.7_25 support
  • P02 – JRE1.7_21 support

that means that following filesets are consistent:

  • wdk/system/ucfinit.jar, wdk/fileselector/fileSelector.jar, wdk/contentXfer from P11 for JRE1.7_51
  • wdk/system/ucfinit.jar, wdk/fileselector/fileSelector.jar, wdk/contentXfer from P10 for JRE1.7_45
  • wdk/system/ucfinit.jar, wdk/fileselector/fileSelector.jar, wdk/contentXfer from P06 for JRE1.7_25
  • wdk/system/ucfinit.jar, wdk/fileselector/fileSelector.jar, wdk/contentXfer from P04 for JRE1.7_21

for filesets 2-4 I created following structure inside wdk directory:

JRE17_21  
├── contentXfer  
│  ├── All-MB.jar  
│  ├── ES1_MRE.exe  
│  ├── ExJNIAPI.dll  
│  ├── ExJNIAPIGateway.jar  
│  ├── jacob.dll  
│  ├── jacob.jar  
│  ├── libMacOSXForkerIO.jnilib  
│  ├── MacOSXForker.jar  
│  ├── mac_utilities.jar  
│  ├── ucf-ca-office-auto.jar  
│  ├── ucf-client-installer.zip  
│  └── UCFWin32JNI.dll  
├── fileselector  
│  └── fileSelector.jar  
└── system  
    └── ucfinit.jar  
  
JRE17_25  
├── contentXfer  
│  ├── All-MB.jar  
│  ├── ES1_MRE.exe  
│  ├── ExJNIAPI.dll  
│  ├── ExJNIAPIGateway.jar  
│  ├── jacob.dll  
│  ├── jacob.jar  
│  ├── libMacOSXForkerIO.jnilib  
│  ├── MacOSXForker.jar  
│  ├── mac_utilities.jar  
│  ├── ucf-ca-office-auto.jar  
│  ├── ucf-client-installer.zip  
│  └── UCFWin32JNI.dll  
├── fileselector  
│  └── fileSelector.jar  
└── system  
    └── ucfinit.jar  
  
JRE17_45  
├── contentXfer  
│  ├── All-MB.jar  
│  ├── ES1_MRE.exe  
│  ├── ExJNIAPI.dll  
│  ├── ExJNIAPIGateway.jar  
│  ├── jacob.dll  
│  ├── jacob.jar  
│  ├── libMacOSXForkerIO.jnilib  
│  ├── MacOSXForker.jar  
│  ├── mac_utilities.jar  
│  ├── ucf-ca-office-auto.jar  
│  ├── ucf-client-installer.zip  
│  └── UCFWin32JNI.dll  
├── fileselector  
│  └── fileSelector.jar  
└── system  
    └── ucfinit.jar

then I put urlrewritefilter-4.0.3.jar (http://tuckey.org/urlrewrite/) into WEB-INF/lib directory and added following lines to web.xml:

<filter>  
    <filter-name>UrlRewriteFilter</filter-name>  
    <filter-class>org.tuckey.web.filters.urlrewrite.UrlRewriteFilter</filter-class>  
    <init-param>  
        <param-name>modRewriteConf</param-name>  
        <param-value>true</param-value>  
    </init-param>  
</filter>  
  
<filter-mapping>  
    <filter-name>UrlRewriteFilter</filter-name>  
    <url-pattern>/*</url-pattern>  
    <dispatcher>REQUEST</dispatcher>  
    <dispatcher>FORWARD</dispatcher>  
</filter-mapping>

and finally put following .htaccess file into WEB-INF directory:

RewriteCond  %{HTTP_USER_AGENT}  Java/1\.7\.0_21  
RewriteRule  ^/(_[^/]/[^/]*?-[^/]*?/)?wdk/(system/ucfinit\.jar|fileselector/fileSelector\.jar|contentXfer/(All-MB\.jar|ES1_MRE\.exe|ExJNIAPI\.dll|ExJNIAPIGateway\.jar|jacob\.dll|jacob\.jar|libMacOSXForkerIO\.jnilib|libUCFLinuxGNOME\.so|libUCFLinuxJNI\.so|libUCFLinuxKDE\.so|libUCFSolarisGNOME\.so|libUCFSolarisJNI\.so|MacOSXForker\.jar|mac_utilities\.jar|ucf-ca-office-auto\.jar|ucf-client-installer\.zip|UCFWin32JNI\.dll))$  /wdk/JRE17_21/$2  [PT,L]  
      
RewriteCond  %{HTTP_USER_AGENT}  Java/1\.7\.0_25
RewriteRule  ^/(_[^/]/[^/]*?-[^/]*?/)?wdk/(system/ucfinit\.jar|fileselector/fileSelector\.jar|contentXfer/(All-MB\.jar|ES1_MRE\.exe|ExJNIAPI\.dll|ExJNIAPIGateway\.jar|jacob\.dll|jacob\.jar|libMacOSXForkerIO\.jnilib|libUCFLinuxGNOME\.so|libUCFLinuxJNI\.so|libUCFLinuxKDE\.so|libUCFSolarisGNOME\.so|libUCFSolarisJNI\.so|MacOSXForker\.jar|mac_utilities\.jar|ucf-ca-office-auto\.jar|ucf-client-installer\.zip|UCFWin32JNI\.dll))$  /wdk/JRE17_25/$2  [PT,L]
      
RewriteCond  %{HTTP_USER_AGENT}  Java/1\.7\.0_45
RewriteRule  ^/(_[^/]/[^/]*?-[^/]*?/)?wdk/(system/ucfinit\.jar|fileselector/fileSelector\.jar|contentXfer/(All-MB\.jar|ES1_MRE\.exe|ExJNIAPI\.dll|ExJNIAPIGateway\.jar|jacob\.dll|jacob\.jar|libMacOSXForkerIO\.jnilib|libUCFLinuxGNOME\.so|libUCFLinuxJNI\.so|libUCFLinuxKDE\.so|libUCFSolarisGNOME\.so|libUCFSolarisJNI\.so|MacOSXForker\.jar|mac_utilities\.jar|ucf-ca-office-auto\.jar|ucf-client-installer\.zip|UCFWin32JNI\.dll))$  /wdk/JRE17_45/$2  [PT,L]

And now I have webtop build that is compatible with four JRE security baselines, what about yours?!

Start Java Method Server Properly

Post was originally published on ECN.

Long time ago I noticed some misbehavior in Content Server: if Java Method Server was down for a long time and then started it takes significant time for Content Server to understand that JMS is up and running. In such situation I have used one of two following “solutions” to resume proper work of workflow methods:

  • restart Content Server
  • disable and enable JMS in DA (DA->Admin->basic configuration->Java Method Server)

Both solutions were wrong! Content Server uses following stupid algorithm to check JMS availability:

  • if CS finds out that JMS is unreachable it executes JMSHealthChecker method
  • if execution of JMSHealthChecker was unsuccessful CS increments check interval
  • successful execution of JMSHealthChecker method resets check interval

Let’s check what requests does Content Server send when JMS is unavailable using nc utility and debugging capabilities of DUMP_JMS_CONFIG_LIST RPC-command:

~]$ while `true`;do date; nc -l 9080 < /dev/null; done  
Tue Feb  4 06:06:42 MSK 2014  
POST /bpm/servlet/DoMethod HTTP/1.1  
User-Agent: Documentum Server 6.7.1240.0300  Linux.Oracle (HTTP Client)  
Host: docu67dev01:9080  
Connection: close  
Content-Type: application/x-www-form-urlencoded  
Content-Length: 502  
  
... method_verb=com.documentum.bpm.rtutil.JMSHealthCheckMethod ...  
Tue Feb  4 06:06:48 MSK 2014  
POST /bpm/servlet/DoMethod HTTP/1.1  
User-Agent: Documentum Server 6.7.1240.0300  Linux.Oracle (HTTP Client)  
Host: docu67dev01:9080  
Connection: close  
Content-Type: application/x-www-form-urlencoded  
Content-Length: 502  
  
... method_verb=com.documentum.bpm.rtutil.JMSHealthCheckMethod ...  
Tue Feb  4 06:07:48 MSK 2014  
POST /bpm/servlet/DoMethod HTTP/1.1  
User-Agent: Documentum Server 6.7.1240.0300  Linux.Oracle (HTTP Client)  
Host: docu67dev01:9080  
Connection: close  
Content-Type: application/x-www-form-urlencoded  
Content-Length: 502  
  
... method_verb=com.documentum.bpm.rtutil.JMSHealthCheckMethod ...  
Tue Feb  4 06:09:48 MSK 2014  
POST /bpm/servlet/DoMethod HTTP/1.1  
User-Agent: Documentum Server 6.7.1240.0300  Linux.Oracle (HTTP Client)  
Host: docu67dev01:9080  
Connection: close  
Content-Type: application/x-www-form-urlencoded  
Content-Length: 502  
  
... method_verb=com.documentum.bpm.rtutil.JMSHealthCheckMethod ...  
Tue Feb  4 06:13:48 MSK 2014

Note incrementing intervals between timestamps: 06:06:48 -> 06:07:48 -> 06:09:48 -> 06:13:48

API> apply,c,,DUMP_JMS_CONFIG_LIST  
...  
q0  
API> next,c,q0  
...  
OK  
API> dump,c,q0  
...  
USER ATTRIBUTES  
  
  jms_list_last_refreshed         : Tue Feb  4 05:50:46 2014  
  incr_wait_time_on_failure       : 30  
  max_wait_time_on_failure        : 3600  
  current_jms_index               : 0  
  jms_config_id                [0]: 0801d92080000b65  
  jms_config_name              [0]: JMS docu67dev01:9080 for repo.repo  
  server_config_id             [0]: 3d01d92080000102  
  server_config_name           [0]: repo  
  jms_to_cs_proximity          [0]: 1  
  is_disabled_in_docbase       [0]: F  
  is_marked_dead_in_cache      [0]: T  
  intended_purpose             [0]: DM_JMS_PURPOSE_DEFAULT_EMBEDDED_JMS  
  last_failure_time            [0]: Tue Feb  4 06:06:48 2014  
  next_retry_time              [0]: Tue Feb  4 06:07:48 2014  
  failure_count                [0]: 2  
  
SYSTEM ATTRIBUTES  
  
APPLICATION ATTRIBUTES  
  
INTERNAL ATTRIBUTES  
  
API> close,c,q0  
...  
OK   
API> apply,c,,DUMP_JMS_CONFIG_LIST  
...  
q0  
API> next,c,q0  
...  
OK  
API> dump,c,q0  
...  
USER ATTRIBUTES  
  
  jms_list_last_refreshed         : Tue Feb  4 05:50:46 2014  
  incr_wait_time_on_failure       : 30  
  max_wait_time_on_failure        : 3600  
  current_jms_index               : 0  
  jms_config_id                [0]: 0801d92080000b65  
  jms_config_name              [0]: JMS docu67dev01:9080 for repo.repo  
  server_config_id             [0]: 3d01d92080000102  
  server_config_name           [0]: repo  
  jms_to_cs_proximity          [0]: 1  
  is_disabled_in_docbase       [0]: F  
  is_marked_dead_in_cache      [0]: T  
  intended_purpose             [0]: DM_JMS_PURPOSE_DEFAULT_EMBEDDED_JMS  
  last_failure_time            [0]: Tue Feb  4 06:07:48 2014  
  next_retry_time              [0]: Tue Feb  4 06:09:48 2014  
  failure_count                [0]: 3  
  
SYSTEM ATTRIBUTES  
  
APPLICATION ATTRIBUTES  
  
INTERNAL ATTRIBUTES  
  
API> close,c,q0  
...  
OK  
  
  
API> apply,c,,DUMP_JMS_CONFIG_LIST  
...  
q0  
API> next,c,q0  
...  
OK  
API> dump,c,q0  
...  
USER ATTRIBUTES  
  
  jms_list_last_refreshed         : Tue Feb  4 05:50:46 2014  
  incr_wait_time_on_failure       : 30  
  max_wait_time_on_failure        : 3600  
  current_jms_index               : 0  
  jms_config_id                [0]: 0801d92080000b65  
  jms_config_name              [0]: JMS docu67dev01:9080 for repo.repo  
  server_config_id             [0]: 3d01d92080000102  
  server_config_name           [0]: repo  
  jms_to_cs_proximity          [0]: 1  
  is_disabled_in_docbase       [0]: F  
  is_marked_dead_in_cache      [0]: T  
  intended_purpose             [0]: DM_JMS_PURPOSE_DEFAULT_EMBEDDED_JMS  
  last_failure_time            [0]: Tue Feb  4 06:09:48 2014  
  next_retry_time              [0]: Tue Feb  4 06:13:48 2014  
  failure_count                [0]: 4  
  
SYSTEM ATTRIBUTES  
  
APPLICATION ATTRIBUTES  
  
INTERNAL ATTRIBUTES  
  
API> close,c,q0  
...  
OK

Note next_retry_time timestamps: 06:06:48 -> 06:07:48 -> 06:09:48 -> 06:13:48 – the same as in nc output.

Initially I thought that Content Server increments check interval by 30 seconds (see incr_wait_time_on_failure in output of DUMP_JMS_CONFIG_LIST and ), but it seems behavior depends on whether Content Server is able to establish TCP connection or not – if you just shutdown Java Method Server and execute DUMP_JMS_CONFIG_LIST you will find that check interval is incremented by 30 seconds but in my experiment (I used nc to dump http traffic, so Content Server is able to establish TCP connection) I noticed another behavior.

Now about the proper way to reset check interval:

API> apply,c,,TIME  
...  
q0  
API> next,c,q0  
...  
OK  
API> get,c,q0,result  
...  
2/4/2014 06:39:28  
API> close,c,q0  
...  
OK  
API> apply,c,,DUMP_JMS_CONFIG_LIST  
...  
q0  
API> next,c,q0  
...  
OK  
API> dump,c,q0  
...  
USER ATTRIBUTES  
  
  jms_list_last_refreshed         : Tue Feb  4 05:50:46 2014  
  incr_wait_time_on_failure       : 30  
  max_wait_time_on_failure        : 3600  
  current_jms_index               : 0  
  jms_config_id                [0]: 0801d92080000b65  
  jms_config_name              [0]: JMS docu67dev01:9080 for repo.repo  
  server_config_id             [0]: 3d01d92080000102  
  server_config_name           [0]: repo  
  jms_to_cs_proximity          [0]: 1  
  is_disabled_in_docbase       [0]: F  
  is_marked_dead_in_cache      [0]: T  
  intended_purpose             [0]: DM_JMS_PURPOSE_DEFAULT_EMBEDDED_JMS  
  last_failure_time            [0]: Tue Feb  4 06:37:48 2014  
  next_retry_time              [0]: Tue Feb  4 07:09:48 2014  
  failure_count                [0]: 7  
  
SYSTEM ATTRIBUTES  
  
APPLICATION ATTRIBUTES  
  
INTERNAL ATTRIBUTES  
  
API> close,c,q0  
...  
OK  
API> apply,c,,REFRESH_JMS_CONFIG_LIST  
...  
q0  
API> next,c,q0  
...  
OK  
API> get,c,q0,result  
...  
T  
API> close,c,q0  
...  
OK  
-- previous execution of JMSHealthCheckMethod at 06:37:48  
POST /bpm/servlet/DoMethod HTTP/1.1  
User-Agent: Documentum Server 6.7.1240.0300  Linux.Oracle (HTTP Client)  
Host: docu67dev01:9080  
Connection: close  
Content-Type: application/x-www-form-urlencoded  
Content-Length: 502  
  
... method_verb=com.documentum.bpm.rtutil.JMSHealthCheckMethod ...  
Tue Feb  4 06:37:48 MSK 2014  
  
  
-- rescheduled execution of JMSHealthCheckMethod at 06:43:18 (not 07:09:48!!!)  
POST /bpm/servlet/DoMethod HTTP/1.1  
User-Agent: Documentum Server 6.7.1240.0300  Linux.Oracle (HTTP Client)  
Host: docu67dev01:9080  
Connection: close  
Content-Type: application/x-www-form-urlencoded  
Content-Length: 502  
  
... method_verb=com.documentum.bpm.rtutil.JMSHealthCheckMethod ...  
Tue Feb  4 06:42:48 MSK 2014  
POST /bpm/servlet/DoMethod HTTP/1.1  
User-Agent: Documentum Server 6.7.1240.0300  Linux.Oracle (HTTP Client)  
Host: docu67dev01:9080  
Connection: close  
Content-Type: application/x-www-form-urlencoded  
Content-Length: 502  
  
... method_verb=com.documentum.bpm.rtutil.JMSHealthCheckMethod ...  
Tue Feb  4 06:43:18 MSK 2014  
POST /bpm/servlet/DoMethod HTTP/1.1  
User-Agent: Documentum Server 6.7.1240.0300  Linux.Oracle (HTTP Client)  
Host: docu67dev01:9080  
Connection: close  
Content-Type: application/x-www-form-urlencoded  
Content-Length: 502  
  
... method_verb=com.documentum.bpm.rtutil.JMSHealthCheckMethod ...  
Tue Feb  4 06:44:18 MSK 2014  

Execution of REFRESH_JMS_CONFIG_LIST RPC command resets JMSHealthCheckMethod check interval.

DM_GROUP_LIST_LIMIT_TEMP_TBL vs Oracle Database

Content Server recognizes three environment variables which control the manner of ACL checks in DQL queries, these variables are:

DM_GROUP_LIST_LIMIT – sets the upper limit (default is 250) of groups, user belongs to, after which CS performs ACL checks using following manner:

AND ((EXISTS
   (SELECT 1
    FROM dm_acl_s ACL_S0, dm_acl_r ACL_R
   WHERE ACL_S0.r_object_id = ACL_R.r_object_id
     AND dm_folder.acl_domain = ACL_S0.owner_name
     AND dm_folder.acl_name = ACL_S0.object_name
     AND (( ACL_R.r_accessor_name IN
         ('dmadmin', 'dm_world')
       OR (ACL_R.r_is_group = 1
         AND (EXISTS
           (SELECT 1
            FROM dm_group_r gr1, dm_group_r gr2
           WHERE gr1.i_nondyn_supergroups_names =
              ACL_R.r_accessor_name
             AND gr1.r_object_id =
                gr2.r_object_id
             AND gr2.users_names = 'dmadmin'
             AND gr1.i_nondyn_supergroups_names
                IS NOT NULL
            UNION ALL
            SELECT 1
            FROM dm_group_r gr1, dm_group_r gr2
           WHERE gr1.i_nondyn_supergroups_names =
              ACL_R.r_accessor_name
             AND gr1.r_object_id =
                gr2.r_object_id
             AND gr2.groups_names =
                'dm_world'
             AND gr1.i_nondyn_supergroups_names
                IS NOT NULL)))
       OR (ACL_R.r_accessor_name = 'dm_owner'))
        AND (( ACL_R.r_permit_type = 0
          OR ACL_R.r_permit_type IS NULL)
         AND (((ACL_R.r_accessor_permit >= 6))))))))

DM_GROUP_LIST_LIMIT_TEMP_TBL – if set to T, CS stores user’s groups in “temporary” (“temporary” here does not mean true temporary tables, but regular tables which persist in database during some period of time) table and performs security checks using following manner:

AND ((EXISTS
   (SELECT 1
    FROM dm_acl_s ACL_S0, dm_acl_r ACL_R
   WHERE ACL_S0.r_object_id = ACL_R.r_object_id
     AND dm_folder.acl_domain = ACL_S0.owner_name
     AND dm_folder.acl_name = ACL_S0.object_name
     AND (( ACL_R.r_accessor_name IN
         ('dmadmin', 'dm_world')
       OR (ACL_R.r_is_group = 1
         AND (EXISTS
           (SELECT 1
            FROM dmdql80112100000
           WHERE ACL_R.r_accessor_name =
              group_name)))
       OR (ACL_R.r_accessor_name = 'dm_owner'))
        AND (( ACL_R.r_permit_type = 0
          OR ACL_R.r_permit_type IS NULL)
         AND (((ACL_R.r_accessor_permit >= 6))))))))

DM_LEFT_OUTER_JOIN_FOR_ACL – seems does not have effect for Oracle docbases, so I don’t know how CS performs ACL checks in this case.

The problem is when DM_GROUP_LIST_LIMIT_TEMP_TBL is in effect, Content Server permanently creates and drops “temporary” tables, but due to recycle bin feature introduced in Oracle 10g, oracle does not really drop those temporary tables but moves them to recycle bin which leads to the following performance impact: when Oracle runs out of free space it tries to reclaim space by purging objects from recycle bin and database inserts get stuck on “enq: CR – block range reuse ckpt”. So, if you are going to use DM_GROUP_LIST_LIMIT_TEMP_TBL feature disable recycle bin in Oracle to avoid performance troubles.

D2-Config vs IE11

In spite of the fact that D2 has no practical interest for me (it has a good collection of anti-patters though), yesterday one of my skypemates asked me how to make D2-Config work in IE11. The problem is EMC added http-equiv attributes into D2 pages but forgot about D2-Config, so D2-Config does not work in IE11.

I know three options to resolve this issue:

  1. If you use apache httpd as reverse proxy it is enough to add Header set X-UA-Compatible “IE=EmulateIE7” into httpd.conf
  2. You can achieve the same on application server side using urlrewritefilter, something like:
    web.xml:
    <filter>
        <filter-name>UrlRewriteFilter</filter-name>
        <filter-class>org.tuckey.web.filters.urlrewrite.UrlRewriteFilter</filter-class>
    </filter>
    <filter-mapping>
        <filter-name>UrlRewriteFilter</filter-name>
        <url-pattern>/*</url-pattern>
        <dispatcher>REQUEST</dispatcher>
        <dispatcher>FORWARD</dispatcher>
    </filter-mapping>
    
    urlrewrite.xml:
    <?xml version="1.0" encoding="utf-8"?>
    <!DOCTYPE urlrewrite PUBLIC "-//tuckey.org//DTD UrlRewrite 4.0//EN" 
          "http://www.tuckey.org/res/dtds/urlrewrite4.0.dtd">
    <urlrewrite>
        <rule>
            <set type="response-header" name="X-UA-Compatible">IE=EmulateIE7</set>
        </rule>
    </urlrewrite>
    
  3. And finally, you can code your own filter:
    import javax.servlet.*;
    import javax.servlet.http.HttpServletResponse;
    import java.io.IOException;
    
    public class IEFilter implements Filter {
    
        @Override
        public void doFilter(ServletRequest request, ServletResponse response, FilterChain chain) 
                throws IOException, ServletException {
            HttpServletResponse res = (HttpServletResponse) response;
            res.addHeader("X-UA-Compatible", "IE=EmulateIE7");
            chain.doFilter(request, response);
        }
    
        @Override
        public void destroy() {
        
        }
    
        @Override
        public void init(FilterConfig filterConfig) throws ServletException {
        
        }
        
    }

Thread Dump JSP

I was trying to google first and found some implementations:

unfortunately, no one displays lock information properly, so I wrote my own.

<%@ page import="java.io.IOException" %>
<%@ page import="java.io.PrintWriter" %>
<%@ page import="java.lang.management.LockInfo" %>
<%@ page import="java.lang.management.ManagementFactory" %>
<%@ page import="java.lang.management.MonitorInfo" %>
<%@ page import="java.lang.management.ThreadInfo" %>
<%@ page import="java.lang.management.ThreadMXBean" %>
<%@ page import="java.util.ArrayList" %>
<%@ page import="java.util.Collections" %>
<%@ page import="java.util.List" %>

<%@ page contentType="text/html;charset=UTF-8" language="java" %>

<%!
    public class MyThreadInfo implements Comparable<MyThreadInfo> {

        private final long threadID;

        private final ThreadInfo threadInfo;

        private final String name;

        public MyThreadInfo(ThreadInfo info) {
            threadInfo = info;
            name = info.getThreadName();
            threadID = info.getThreadId();
        }

        public int compareTo(MyThreadInfo o) {
            String myName = name + threadID;
            String yourName = o.name + o.threadID;
            return myName.compareTo(yourName);
        }

    }

    public List<MyThreadInfo> getThreads(boolean deadlock) {
        ThreadMXBean tmb = ManagementFactory.getThreadMXBean();
        ArrayList<MyThreadInfo> threads = new ArrayList<MyThreadInfo>();
        long[] threadIds = null;
        if (deadlock) {
            threadIds = tmb.findDeadlockedThreads();
        } else {
            threadIds = tmb.getAllThreadIds();
        }
        if (threadIds == null) {
            return Collections.emptyList();
        }
        for (ThreadInfo info : tmb.getThreadInfo(threadIds, true, true)) {
            if (info != null) {
                threads.add(new MyThreadInfo(info));
            }

        }
        Collections.sort(threads);
        return threads;
    }

    public String toString(ThreadInfo threadInfo) {
        StringBuilder sb = new StringBuilder("\"" + threadInfo.getThreadName() + "\"" +
                " Id=" + threadInfo.getThreadId() + " " +
                threadInfo.getThreadState());
        if (threadInfo.getLockName() != null) {
            sb.append(" on ").append(threadInfo.getLockName());
        }
        if (threadInfo.getLockOwnerName() != null) {
            sb.append(" owned by \"").append(threadInfo.getLockOwnerName())
                    .append("\" Id=").append(threadInfo.getLockOwnerId());
        }
        if (threadInfo.isSuspended()) {
            sb.append(" (suspended)");
        }
        if (threadInfo.isInNative()) {
            sb.append(" (in native)");
        }
        sb.append('\n');
        StackTraceElement[] stackTrace = threadInfo.getStackTrace();
        for (int i = 0, n = stackTrace.length; i < n; i++) {
            StackTraceElement ste = stackTrace[i];
            sb.append("\tat ").append(ste.toString());
            sb.append('\n');
            if (i == 0 && threadInfo.getLockInfo() != null) {
                Thread.State ts = threadInfo.getThreadState();
                switch (ts) {
                    case BLOCKED:
                        sb.append("\t-  blocked on ").append(threadInfo.getLockInfo());
                        sb.append('\n');
                        break;
                    case WAITING:
                        sb.append("\t-  waiting on ").append(threadInfo.getLockInfo());
                        sb.append('\n');
                        break;
                    case TIMED_WAITING:
                        sb.append("\t-  waiting on ").append(threadInfo.getLockInfo());
                        sb.append('\n');
                        break;
                    default:
                }
            }

            for (MonitorInfo mi : threadInfo.getLockedMonitors()) {
                if (mi.getLockedStackDepth() == i) {
                    sb.append("\t-  locked ").append(mi);
                    sb.append('\n');
                }
            }
        }

        LockInfo[] locks = threadInfo.getLockedSynchronizers();
        if (locks.length > 0) {
            sb.append("\n\tNumber of locked synchronizers = ").append(locks.length);
            sb.append('\n');
            for (LockInfo li : locks) {
                sb.append("\t- ").append(li);
                sb.append('\n');
            }
        }
        sb.append('\n');
        return sb.toString();
    }

    public void printThreads(List<MyThreadInfo> threads, JspWriter out) throws IOException {
        out.println("<br><b>Total threads: " + threads.size() + "</b>");
        for (MyThreadInfo mti : threads) {
            out.println("<pre>");
            out.print(toString(mti.threadInfo));
            out.println("</pre>");
        }
    }

%>
<html>
<body>
<table>
    <tr>
        <td><b>Date:</b></td>
        <td><%=new java.util.Date()%>
        </td>
    </tr>
</table>

<%
    try {
        printThreads(getThreads(false), out);
        List<MyThreadInfo> deadLockedThreads = getThreads(true);
        if (deadLockedThreads != null && !deadLockedThreads.isEmpty()) {
            out.print("<br><b>Found deadlocks:<b>");
            printThreads(deadLockedThreads, out);
        }
    } catch (Exception e) {
        out.print("<pre>");
        e.printStackTrace(new PrintWriter(out, true));
        out.print("</pre>");
    }
%>
</body>
</html>

Здравствуй, зимнее время!

Этот пост навряд ли будет интересен англоязычным пользователям, в особенности тем, чья страна покрывается полностью одним часовым поясом, поэтому пишу на великом и могучем.

Итак, ваше (ага, я уже два года живу в Мельбурне, и у нас сейчас весна, поэтому ваше) правительство в очередной раз подложило свинью всем айтишникам и таки вернуло летнее время взад. Что интересно, так это то, что вендоры подготовили патчи для своих продуктов относительно давно, но коллеги в скайпе активизировались только на этой неделе.

Кратко о том, как управляет временем документум.

Начиная с релиза 6.0 разработчики решили, что хранить в СУБД локальные даты не круто, и начали в базу писать время в UTC, поведение контролируется атрибутом r_normal_tz в dm_server_config: 0 – пишем в UTC, не 0 – пишем локальное время. Кроме этого в dm_server_config есть параметр r_tz_aware который определяет каким образом CS общается с DFC: T – между клиентом и сервером время ходит в формате iso8601 (время передается в UTC), F – в формате “xxx MMM d HH:mm:ss yyyy” (передается локальное время). Вроде бы все логично (хотя при таком подходе страдают отчетные системы, работающие с СУБД напрямую), однако, и тут разработчики умудрились накосячить (здесь пассаж относительно “указанный уровень работы Content Server-а был спроектирован давно” мне оказался так и не ясен, поскольку 6.0 вышел сравнительно недавно):

т.е. кто сидит на версиях ниже 6.7SP1P22 и 6.7SP2P08 постоянно огребает проблемы со временем.

О проблемах в WDK

Собственно, коллеги нарвались на совершенно безумную реазилацию поддержки часовых поясов в WDK. Для начала, примеры систем, которые делаются программистами для людей:

Youtrack:

Jira:

Ну, т.е. все просто – пользователь выставляет в настройках часовой пояс и наступает счастье. Теперь настройки Webtop:

Не густо :(. Как реализованы часовые пояса в Webtop?

в /wdk/redirect.jsp следующий код передает разницу в минутах между часовым поясом клиента и UTC:

    <script type="text/javascript">
        function onLoad() {
            <%--
            // an image.--%>
            var strUrl = new Object;
            var strUrlString = addBrowserIdToURL('<%=strUrl%>');
            strUrl.src = addBrowserIdToURL(strUrlString)
                    + "&<%=IParams.TIME_ZONE_OFFSET%>=" + new Date().getTimezoneOffset();
            navigateToURL(strUrl.src, "redirectForm", window)
        }
    </script>

Далее этот параметр (__dmfTzoff) обрабатывается в com.documentum.web.env.EnvironmentService#notifyRequestStart и вызывается com.documentum.web.common.LocaleService#setTimeZone, где по смещению в минутах разработчики пытаются определить часовой пояс клиента используя следующий забавный алгоритм:

перебираем все часовые пояса из java.util.TimeZone#getAvailableIDs() и выбираем первый, смещение которого совпадает с переданным

И тут нас ждет сюрприз (пример для Калининграда):

import java.util.Calendar;
import java.util.TimeZone;

/**
 * @author Andrey B. Panfilov <andrew@panfilov.tel>
 */
public class Test {

	public static void main(String[] args) throws Exception {
		long offset = 2 * 60 * 1000 * 60;
		Calendar calendar = Calendar.getInstance();
		calendar.set(Calendar.MONTH, calendar.get(Calendar.MONTH) + 1);
		for (String tzid : TimeZone.getAvailableIDs()) {
			TimeZone tz = TimeZone.getTimeZone(tzid);
			if (getOffset(calendar.getTimeInMillis(), tz) == offset) {
				System.out.println("Offset: " + offset + ", TZ: " + tzid);
			}
		}
	}

	private static long getOffset(long timestamp, TimeZone tz) {
		return tz.getOffset(timestamp);
	}

}

результат:

Offset: 7200000, TZ: Africa/Windhoek
Offset: 7200000, TZ: ART
Offset: 7200000, TZ: Africa/Blantyre
Offset: 7200000, TZ: Africa/Bujumbura
...
Offset: 7200000, TZ: Europe/Tiraspol
Offset: 7200000, TZ: Europe/Uzhgorod
Offset: 7200000, TZ: Europe/Vilnius
Offset: 7200000, TZ: Europe/Zaporozhye
Offset: 7200000, TZ: Israel
Offset: 7200000, TZ: Libya
Offset: 7200000, TZ: Turkey
Offset: 7200000, TZ: Europe/Kaliningrad

Первым в списке идет Africa/Windhoek, соответственно, для пользователей Калининграда, работающих с Webtop, раположенном в Москве, Webtop будет считать, что часовой пояс у пользователя – Africa/Windhoek, а там сейчас не зима, а лето, и у пользователя летние даты будут отображаться со смещением на два часа вперед.

Собственно, я, не особо долго думая сделал такой патч для com.documentum.web.common.LocaleService:

    private static List<String> s_preferredTimeZones = null;

    private static final String PREFERRED_TIMEZONES_CONFIG_ELEMENT 
                                  = "application.preferred_timezones";

    private static final String TIMEZONE_CONFIG_ELEMENT = "timezone";

    public static synchronized List<String> getPreferredTimezones() {
        if (s_preferredTimeZones == null) {
            s_preferredTimeZones = new ArrayList<String>();
            IConfigLookup lookup = ConfigService.getConfigLookup();
            IConfigElement configTimeZones = lookup.lookupElement(
                    PREFERRED_TIMEZONES_CONFIG_ELEMENT, getContext());
            if (configTimeZones == null) {
                return Collections.unmodifiableList(s_preferredTimeZones);
            }
            for (Iterator iter = configTimeZones
                    .getChildElements(TIMEZONE_CONFIG_ELEMENT); iter.hasNext();) {
                IConfigElement timeZone = (IConfigElement) iter.next();
                String name = timeZone.getValue();
                if (name != null) {
                    s_preferredTimeZones.add(name);
                }
            }

        }
        return Collections.unmodifiableList(s_preferredTimeZones);
    }

    public static void setTimeZone(int clientTzOffset) {
        if (!hasClientTimeZoneEnabled()) {
            return;
        }
        if (SessionState.getAttribute(TIMEZONE_SESSION_VAR) != null) {
            return;
        }

        clientTzOffset = -clientTzOffset * 60 * 1000;

        long now = System.currentTimeMillis();
        TimeZone clientTz = TimeZone.getDefault();
        if (clientTz.getOffset(now) != clientTzOffset) {
            List<String> preferredTimeZones = getPreferredTimezones();
            List<TimeZone> zones = new ArrayList<TimeZone>();
            boolean found = false;
            for (String tzid : TimeZone.getAvailableIDs()) {
                TimeZone tz = TimeZone.getTimeZone(tzid);
                if (tz.getOffset(now) != clientTzOffset) {
                    continue;
                }
                if (preferredTimeZones.contains(tzid)) {
                    clientTz = tz;
                    found = true;
                    break;
                }
                zones.add(tz);
            }
            if (!found && !zones.isEmpty()) {
                clientTz = zones.get(0);
            }
        }
        SessionState.setAttribute(TIMEZONE_SESSION_VAR, clientTz);
        //Забавно, но две строчки ниже никогда не работают, потому что первый вызов
        //идет со страницы логина, поэтому запись в RepositoryPreferencesStore не 
        //осуществляется - логин пользователя не известен, а потом до этого места
        //не доходит из-за проверок SessionState
        IPreferenceStore prefs = PreferenceService.getPreferenceStore();
        prefs.writeString(TIMEZONE_PREFERENCE, clientTz.getID());
        if (Trace.LOCALESERVICE) {
            Trace.println("LocaleService: Time zone set to "
                    + clientTz.getDisplayName());
        }
    }

и пишу в app.xml следующее:

            <preferred_timezones>
                <timezone>Europe/Kaliningrad</timezone>
                <timezone>Europe/Moscow</timezone>
            </preferred_timezones>

Вообще не плохо было бы подключить ICU и написать более адекватный алгоритм, но лень.