setFile API usage limitis

Today I received a comment for Ingestion rates blogpost:

object.setFile(“path to file”); ‘path to file’ is limited to 255 characters in 6.x, I am not sure in 7.x.
You might have already noticed it but thought of sharing with you.

Well, that is true that the problem exists, but that is not true that it is a some kind of internal limit – it is just a bug in DFC. The first time when I faced with such misbehaviour was a project, when I was helping customer to upgrade Documentum from 6.5 to 6.7 – I always “like” such projects because they always cause a lot of stress: customers always expect that higher version numbers mean more product stability and maturity, which is not true in case of Documentum, moreover, customers are always happy to treat you like a whipping-boy and pin old bugs on you. Anyway, that time the error was sounding like “webtop import does not work” and corresponding exception was looking like:

THREAD: Timer-8262; MSG: [DFC_OBJECT_BADATTRVALUE] value is too big for attribute 'set_file'. Value UTF-8 length is 388. Maximum length is 255.; ERRORCODE: ff; NEXT: null
 at com.documentum.fc.client.impl.typeddata.Attribute.validateValueLength(Attribute.java:285)
 at com.documentum.fc.client.impl.typeddata.AbstractTypedData.validateAndConvertIfNecessary(AbstractTypedData.java:325)
 at com.documentum.fc.client.impl.typeddata.AbstractTypedData.setRepeatingString(AbstractTypedData.java:419)
 at com.documentum.fc.client.DfTypedObject.setRepeatingStringInternal(DfTypedObject.java:1882)
 at com.documentum.fc.client.DfTypedObject.setRepeatingStringRaw(DfTypedObject.java:1867)
 at com.documentum.fc.client.DfTypedObject$AbstractAttributeFilterImpl.set(DfTypedObject.java:3446)
 at com.documentum.fc.client.content.impl.Content$SetFileHandler.set(Content.java:1515)
 at com.documentum.fc.client.DfTypedObject.doSetString(DfTypedObject.java:1441)
 at com.documentum.fc.client.DfTypedObject.setString(DfTypedObject.java:1418)
 at com.documentum.fc.client.content.impl.Content.setSetFile(Content.java:758)
 at com.documentum.fc.client.content.impl.Content.setContentSaver(Content.java:77)
 at com.documentum.fc.client.content.impl.Content___PROXY.setContentSaver(Content___PROXY.java)
 at com.documentum.fc.client.content.impl.ContentManager.newContent(ContentManager.java:352)
 at com.documentum.fc.client.content.impl.ContentManager.makeContentInternal(ContentManager.java:522)
 at com.documentum.fc.client.content.impl.ContentManager.addContentInternal(ContentManager.java:486)
 at com.documentum.fc.client.content.impl.ContentManager.setFile(ContentManager.java:480)
 at com.documentum.fc.client.DfSysObject.setContentSaverManager(DfSysObject.java:4343)
 at com.documentum.fc.client.DfDocument___PROXY.setContentSaverManager(DfDocument___PROXY.java)
 at com.documentum.operations.nodeactions.inbound.DfSetContentFile.setContentFile(DfSetContentFile.java:253)
 at com.documentum.operations.nodeactions.inbound.DfSetContentFile.execute(DfSetContentFile.java:37)
 at com.documentum.operations.steps.impl.OperationStep.executeStep(OperationStep.java:163)
 at com.documentum.operations.steps.impl.OperationStep.execute(OperationStep.java:41)
 at com.documentum.operations.impl.OperationExecutionEngine.execute(OperationExecutionEngine.java:51)
 at com.documentum.operations.DfOperation.execute(DfOperation.java:401)
 at com.documentum.operations.inbound.impl.InboundOperation.execute(InboundOperation.java:104)
 at com.documentum.operations.inbound.DfImportOperation.execute(DfImportOperation.java:96)
 at com.documentum.web.contentxfer.DFCOperationSupport.execute(DFCOperationSupport.java:61)
 at com.documentum.web.contentxfer.ContentTransferService.execute(ContentTransferService.java:377)
 at com.documentum.web.contentxfer.JobAdapter.execute(JobAdapter.java:108)
 at com.documentum.job.async.AsyncJobManager$AsyncTimerJob.executeJob(AsyncJobManager.java:236)
 at com.documentum.job.async.AsyncJobManager$AsyncTimerJob.run(AsyncJobManager.java:215)
 at java.util.TimerThread.mainLoop(Timer.java:512)
 at java.util.TimerThread.run(Timer.java:462)

If my memory server me right, I had spent about a day to investigate what was the root cause of the problem (“value is too big for attribute ‘set_file’. Value UTF-8 length is 388. Maximum length is 255.” message seems extremely useful, doesn’t it?) and the result of my investigation was:

  1. DFC validates length of attribute values using byte semantic, i.e. com.documentum.fc.client.impl.typeddata.Attribute.validateValueLength method looks like:
    public void validateValueLength(String value) {
    	int attrLength = getLength();
    	String attrName = getName();
    	if (attrLength == 0) {
    		return;
    	}
    	int allowedLength = getAllowedLength(value);
    	if (value.length() <= allowedLength) {
    		return;
    	}
    	throw new ValueLengthException(attrName, value.getBytes("UTF-8").length, attrLength);
    }
    
    public final int getAllowedLength(String value) {
    	int attrLength = getLength();
    	if (attrLength == 0) {
    		return Integer.MAX_VALUE;
    	}
    	return UTF8Util.calculateMaximumAllowedChars(value, attrLength);
    }
    
  2. com.documentum.fc.client.content.impl.Content.setContentSaver method does know about value length limits – it tries to truncate value but does that in wrong way – it has no idea about byte semantic:
    String clientPath = contentSaver.getClientPath();
    int maxLength = getMaxSetSetFileAttrLength();
    if (clientPath != null && clientPath.length() > maxLength) {
    	warn("set_file truncated for object" + getObjectId());
    	clientPath = clientPath.substring(0, maxLength);
    }
    setSetFile(clientPath);
    
  3. Disabling ACS write capabilities mitigated the problem

I have no idea why, but customer preferred to disable ACS write capabilities in webtop over filing a CR to EMC support (Actually, I’m kidding – I do know why).

In your case the minimal reproducible test case is:

/**
 * @author Andrey B. Panfilov <andrey@panfilov.tel>
 */
public class SetFileTest {

	public static final int MAX_LENGTH = 100;

	public static void main(String[] args) throws DfException, IOException {
		IDfSession session = new DfClientX().getLocalClient().newSession("DCTM_DEV",
				new DfLoginInfo("dmadmin", "dmadmin"));
		for (char c : new char[] { 'c', '愛', }) {
			IDfDocument document = (IDfDocument) session.newObject("dm_document");
			document.setFileEx(buildPath(c, 512), "crtext", 0, null);
			document.save();
			IDfContent content = (IDfContent) session.getObject(document.getContentsId());
			System.out.println(content.getString("set_file"));
		}
	}

	private static String buildPath(char c, int length) throws IOException {
		StringBuilder result = new StringBuilder(length);
		int cLen = utfLen(Character.valueOf(c).toString());
		while (utfLen(result) < length) {
			if (MAX_LENGTH - (utfLen(result) % MAX_LENGTH) <= cLen && result.charAt(result.length() - 1) != '/') {
				String path = result.toString();
				new File(path).mkdirs();
				result.append(File.separatorChar);
			}
			result.append(c);
		}
		String path = result.toString();
		File f = new File(path);
		f.createNewFile();
		System.out.println("Path: " + path);
		return path;
	}

	private static int utfLen(CharSequence str) throws IOException {
		return str.toString().getBytes("UTF-8").length;
	}

}
Path: ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc/ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc/ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc/ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc/ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc/cccccccccccc
[main] WARN com.documentum.fc.client.content.impl.Content  - set_file truncated for object06024be980005afb
ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc/ccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc/ccccccccccccccccccccccccccccccccccccccccccccccccccccccc
Path: 愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛/愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛/愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛/愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛/愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛愛/愛愛愛愛
Exception in thread "main" DfAttributeValueException:: THREAD: main; MSG: [DFC_OBJECT_BADATTRVALUE] value is too big for attribute 'set_file'. Value UTF-8 length is 512. Maximum length is 255.; ERRORCODE: ff; NEXT: null
	at com.documentum.fc.client.impl.typeddata.Attribute.validateValueLength(Attribute.java:285)
	at com.documentum.fc.client.impl.typeddata.AbstractTypedData.validateAndConvertIfNecessary(AbstractTypedData.java:325)
	at com.documentum.fc.client.impl.typeddata.AbstractTypedData.setRepeatingString(AbstractTypedData.java:419)
	at com.documentum.fc.client.DfTypedObject.setRepeatingStringInternal(DfTypedObject.java:1882)
	at com.documentum.fc.client.DfTypedObject.setRepeatingStringRaw(DfTypedObject.java:1867)
	at com.documentum.fc.client.DfTypedObject$AbstractAttributeFilterImpl.set(DfTypedObject.java:3446)
	at com.documentum.fc.client.content.impl.Content$SetFileHandler.set(Content.java:1516)
	at com.documentum.fc.client.DfTypedObject.doSetString(DfTypedObject.java:1441)
	at com.documentum.fc.client.DfTypedObject.setString(DfTypedObject.java:1418)
	at com.documentum.fc.client.content.impl.Content.setSetFile(Content.java:759)
	at com.documentum.fc.client.content.impl.Content.setContentSaver(Content.java:77)

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s