importing text file more than 32MB gives error "Caused by: org.bson.BsonSerializationException: Size 30831765 is larger than MaxDocumentSize 16793600."

As we transition to the new Hyland Community, some post links may be temporarily disrupted as we finalize redirects. We apologize for the inconvenience and are working to resolve this as quickly as possible. Details.

Santosh_Yadav

Hi, I am importing text file of size arroung 32MB. While importing file I am getting error as follows: Caused by: org.bson.BsonSerializationException: Size 30831765 is larger than MaxDocumentSize 16793600.

org.nuxeo.ecm.automation.OperationException: Failed to invoke operation FileManager.Import at org.nuxeo.ecm.automation.core.impl.InvokableMethod.invoke(InvokableMethod.java:188) at org.nuxeo.ecm.automation.core.impl.OperationChainCompiler$OperationMethod.invoke(OperationChainCompiler.java:128) at org.nuxeo.ecm.automation.core.impl.OperationChainCompiler$CompiledChainImpl.invoke(OperationChainCompiler.java:196) at org.nuxeo.ecm.automation.core.impl.OperationServiceImpl.run(OperationServiceImpl.java:119) at org.nuxeo.ecm.automation.core.impl.OperationServiceImpl.run(OperationServiceImpl.java:107) at org.nuxeo.ecm.automation.server.jaxrs.batch.BatchManagerComponent.execute(BatchManagerComponent.java:301) at org.nuxeo.ecm.automation.server.jaxrs.batch.BatchManagerComponent.execute(BatchManagerComponent.java:255) at org.nuxeo.ecm.webapp.action.ImportActions.importDocumentsThroughBatchManager(ImportActions.java:295) at org.nuxeo.ecm.webapp.action.ImportActions.importDocuments(ImportActions.java:247) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.seam.util.Reflections.invoke(Reflections.java:22) at org.jboss.seam.intercept.RootInvocationContext.proceed(RootInvocationContext.java:32) at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:56) at org.jboss.seam.transaction.RollbackInterceptor.aroundInvoke(RollbackInterceptor.java:28) at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) at org.jboss.seam.core.BijectionInterceptor.aroundInvoke(BijectionInterceptor.java:79) at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) at org.nuxeo.ecm.platform.ui.web.util.NuxeoExceptionInterceptor.aroundInvoke(NuxeoExceptionInterceptor.java:80) at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) at org.jboss.seam.core.MethodContextInterceptor.aroundInvoke(MethodContextInterceptor.java:44) at org.jboss.seam.intercept.SeamInvocationContext.proceed(SeamInvocationContext.java:68) at org.jboss.seam.intercept.RootInterceptor.invoke(RootInterceptor.java:107) at org.jboss.seam.intercept.JavaBeanInterceptor.interceptInvocation(JavaBeanInterceptor.java:196) at org.jboss.seam.intercept.JavaBeanInterceptor.invoke(JavaBeanInterceptor.java:114) at org.nuxeo.ecm.webapp.action.ImportActions_$$javassist_seam_23.importDocuments(ImportActions$$_javassist_seam_23.java) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.jboss.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:335) at org.jboss.el.util.ReflectionUtil.invokeMethod(ReflectionUtil.java:280) at org.jboss.el.parser.AstMethodSuffix.getValue(AstMethodSuffix.java:59) at org.jboss.el.parser.AstMethodSuffix.invoke(AstMethodSuffix.java:65) at org.jboss.el.parser.AstValue.invoke(AstValue.java:96) at org.jboss.el.MethodExpressionImpl.invoke(MethodExpressionImpl.java:276) at com.sun.faces.facelets.el.TagMethodExpression.invoke(TagMethodExpression.java:105) at org.nuxeo.ecm.platform.ui.web.binding.MetaMethodExpression.invoke(MetaMethodExpression.java:130) at javax.faces.component.MethodBindingMethodExpressionAdapter.invoke(MethodBindingMethodExpressionAdapter.java:87) at com.sun.faces.application.ActionListenerImpl.processAction(ActionListenerImpl.java:102) at javax.faces.component.UICommand.broadcast(UICommand.java:315) at org.nuxeo.ecm.platform.ui.web.component.holder.UIValueHolder.broadcast(UIValueHolder.java:95) at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:790) at javax.faces.component.UIViewRoot.broadcastEvents(UIViewRoot.java:790) at javax.faces.component.UIViewRoot.processApplication(UIViewRoot.java:1282) at com.sun.faces.lifecycle.InvokeApplicationPhase.execute(InvokeApplicationPhase.java:81) at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101) at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:198) at javax.faces.webapp.FacesServlet.service(FacesServlet.java:646) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:83) at org.jboss.seam.web.ExceptionFilter.doFilter(ExceptionFilter.java:64) at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69) at org.jboss.seam.web.RedirectFilter.doFilter(RedirectFilter.java:45) at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69) at org.nuxeo.elasticsearch.web.sync.UIThreadMarker.doFilter(UIThreadMarker.java:48) at org.jboss.seam.servlet.SeamFilter$FilterChainImpl.doFilter(SeamFilter.java:69) at org.jboss.seam.servlet.SeamFilter.doFilter(SeamFilter.java:158) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.nuxeo.ecm.platform.web.common.requestcontroller.filter.NuxeoThreadTrackerFilter.doFilter(NuxeoThreadTrackerFilter.java:43) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.nuxeo.ecm.core.management.jtajca.internal.Log4jWebFilter.doFilter(Log4jWebFilter.java:69) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.nuxeo.ecm.platform.ui.web.rest.FancyURLFilter.doFilter(FancyURLFilter.java:120) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.nuxeo.ecm.platform.web.common.requestcontroller.filter.NuxeoRequestControllerFilter.doFilter(NuxeoRequestControllerFilter.java:146) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.nuxeo.ecm.webdav.service.WIRequestFilter.doFilter(WIRequestFilter.java:61) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.nuxeo.ecm.platform.ui.web.auth.NuxeoAuthenticationFilter.doFilterInternal(NuxeoAuthenticationFilter.java:630) at org.nuxeo.ecm.platform.ui.web.auth.service.NuxeoAuthFilterChain.doFilter(NuxeoAuthFilterChain.java:51) at org.nuxeo.ecm.platform.ui.web.auth.oauth.NuxeoOAuthFilter.doFilter(NuxeoOAuthFilter.java:122) at org.nuxeo.ecm.platform.ui.web.auth.service.NuxeoAuthFilterChain.doFilter(NuxeoAuthFilterChain.java:49) at org.nuxeo.ecm.platform.ui.web.auth.oauth2.NuxeoOAuth2Filter.doFilter(NuxeoOAuth2Filter.java:83) at org.nuxeo.ecm.platform.ui.web.auth.service.NuxeoAuthFilterChain.doFilter(NuxeoAuthFilterChain.java:49) at org.nuxeo.ecm.platform.ui.web.auth.NuxeoAuthenticationFilter.doFilter(NuxeoAuthenticationFilter.java:419) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.nuxeo.ecm.platform.web.common.requestcontroller.filter.NuxeoCorsFilter.doFilter(NuxeoCorsFilter.java:51) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.nuxeo.ecm.platform.web.common.exceptionhandling.NuxeoExceptionFilter.doFilter(NuxeoExceptionFilter.java:67) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.nuxeo.ecm.platform.web.common.encoding.NuxeoEncodingFilter.doFilter(NuxeoEncodingFilter.java:75) at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241) at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208) at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:220) at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122) at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505) at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169) at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103) at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956) at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116) at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:436) at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1078) at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:625) at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:318) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748) Caused by: org.bson.BsonSerializationException: Size 34523331 is larger than MaxDocumentSize 16793600. at org.bson.BsonBinaryWriter.backpatchSize(BsonBinaryWriter.java:367) at org.bson.BsonBinaryWriter.doWriteEndDocument(BsonBinaryWriter.java:122) at org.bson.AbstractBsonWriter.writeEndDocument(AbstractBsonWriter.java:293) at com.mongodb.DBObjectCodec.encode(DBObjectCodec.java:133) at com.mongodb.DBObjectCodec.encode(DBObjectCodec.java:62) at com.mongodb.CompoundDBObjectCodec.encode(CompoundDBObjectCodec.java:48) at com.mongodb.CompoundDBObjectCodec.encode(CompoundDBObjectCodec.java:27) at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:63) at org.bson.codecs.BsonDocumentWrapperCodec.encode(BsonDocumentWrapperCodec.java:29) at com.mongodb.connection.InsertCommandMessage.writeTheWrites(InsertCommandMessage.java:101) at com.mongodb.connection.InsertCommandMessage.writeTheWrites(InsertCommandMessage.java:43) at com.mongodb.connection.BaseWriteCommandMessage.encodeMessageBodyWithMetadata(BaseWriteCommandMessage.java:129) at com.mongodb.connection.RequestMessage.encodeWithMetadata(RequestMessage.java:160) at com.mongodb.connection.WriteCommandProtocol.sendMessage(WriteCommandProtocol.java:220) at com.mongodb.connection.WriteCommandProtocol.execute(WriteCommandProtocol.java:101) at com.mongodb.connection.InsertCommandProtocol.execute(InsertCommandProtocol.java:67) at com.mongodb.connection.InsertCommandProtocol.execute(InsertCommandProtocol.java:37) at com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:168) at com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:289) at com.mongodb.connection.DefaultServerConnection.insertCommand(DefaultServerConnection.java:118) at com.mongodb.operation.InsertOperation.executeCommandProtocol(InsertOperation.java:76) at com.mongodb.operation.BaseWriteOperation$1.call(BaseWriteOperation.java:142) at com.mongodb.operation.BaseWriteOperation$1.call(BaseWriteOperation.java:134) at com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:232) at com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:223) at com.mongodb.operation.BaseWriteOperation.execute(BaseWriteOperation.java:134) at com.mongodb.operation.BaseWriteOperation.execute(BaseWriteOperation.java:61) at com.mongodb.Mongo.execute(Mongo.java:827) at com.mongodb.Mongo$2.execute(Mongo.java:810) at com.mongodb.DBCollection.executeWriteOperation(DBCollection.java:333) at com.mongodb.DBCollection.insert(DBCollection.java:328) at com.mongodb.DBCollection.insert(DBCollection.java:319) at com.mongodb.DBCollection.insert(DBCollection.java:289) at com.mongodb.DBCollection.insert(DBCollection.java:255) at com.mongodb.DBCollection.insert(DBCollection.java:239) at org.nuxeo.ecm.core.storage.mongodb.MongoDBRepository.createStates(MongoDBRepository.java:355) at org.nuxeo.ecm.core.storage.dbs.DBSCachingRepository.createStates(DBSCachingRepository.java:212) at org.nuxeo.ecm.core.storage.dbs.DBSTransactionState.save(DBSTransactionState.java:585) at org.nuxeo.ecm.core.storage.dbs.DBSSession.checkIn(DBSSession.java:532) at org.nuxeo.ecm.core.storage.dbs.DBSDocument.checkIn(DBSDocument.java:485) at org.nuxeo.ecm.core.versioning.StandardVersioningService.doCheckIn(StandardVersioningService.java:339) at org.nuxeo.ecm.core.versioning.VersioningComponent.doCheckIn(VersioningComponent.java:428) at org.nuxeo.ecm.core.api.AbstractSession.checkIn(AbstractSession.java:1740) at org.nuxeo.ecm.core.api.impl.DocumentModelImpl.checkIn(DocumentModelImpl.java:683) at org.nuxeo.ecm.core.versioning.StandardVersioningService.doAutomaticVersioning(StandardVersioningService.java:482) at org.nuxeo.ecm.core.versioning.VersioningComponent.doAutomaticVersioning(VersioningComponent.java:438) at org.nuxeo.ecm.core.api.AbstractSession.createDocument(AbstractSession.java:721) at org.nuxeo.ecm.platform.filemanager.service.extension.AbstractFileImporter.create(AbstractFileImporter.java:227) at org.nuxeo.ecm.platform.filemanager.service.FileManagerService.createDocumentFromBlob(FileManagerService.java:260) at org.nuxeo.ecm.automation.core.operations.services.FileManagerImport.run(FileManagerImport.java:76) at org.nuxeo.ecm.automation.core.operations.services.FileManagerImport.run(FileManagerImport.java:84) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.nuxeo.ecm.automation.core.impl.InvokableMethod.doInvoke(InvokableMethod.java:171) at org.nuxeo.ecm.automation.core.impl.InvokableMethod.invoke(InvokableMethod.java:176)

Kevin_Leturc

Hi,

It seems Nuxeo is trying to store your blob in the document itself, this is a Note specificity. Operation takes the NoteImporter to create the document from blob, which is configured as it:

<plugin name="NoteImporter"  
class="org.nuxeo.ecm.platform.filemanager.service.extension.NoteImporter" order="10">
  <filter>text/plain</filter>
  <filter>text/html</filter>
  <filter>application/xhtml+xml</filter>
  <filter>text/x-web-markdown</filter>
</plugin>

You have several options, set a mime type which is not in the list above and maybe more accurate. Or you can redefine the NoteImporter to exclude the mime type you use from its filter. As it, in both solutions, DefaultFileImporter will be used and a real file will be created on Nuxeo side instead of storing the file content in the document itself.