nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

As we transition to the new Hyland Community, some post links may be temporarily disrupted as we finalize redirects. We apologize for the inconvenience and are working to resolve this as quickly as possible. Details.

Upload large files using Alfresco

amit10may

Hi,
We are using Alfresco CE 3.2 with WebService API from Java webapplication.
Right now the business use case is to upload few documents to Alfresco and its size can vary from 1MB to 100MB.
Our current implementation uses ContentService's
public Content write(Reference node, String property, byte[] content, ContentFormat format) throws RemoteException, ContentFault
API.
Problem with this approach is that the file content has to be loaded completely in byte array and only then can be written to alfresco.
This can become a pain for server esp for large files under heavy load.
Ideally it should be possible to use a stream based approach (as can be done for download content)
I figured out that there is a concept of UploadServlet which can be used to upload content but was not able to understand how to do that by reading the short documentation.

Can someone explain me what is the best way to deal with large file uploads using WebService APIs?
Sample code would be a boon

Regards,
Amit

Find more posts tagged with

Comments

mrogers

WebServices are not suitable for large files. You need to use one of the streaming interfaces.

amit10may

Wow..I did not expect such a quick reply.. so thanks a lot

Can you please give me some pointer to where I can look for help on streaming interface?
I want to integrate it with my Java web application (in fact spring based services are client of the alfresco api)

junieboy

Hi Amit,

Could I ask you in which javadoc would I find the org.alfresco.webservices classes? I am currently using Alfresco CE 3.0 and am not able to see any javadoc for the NamedValue, CMLCreate, UpdateResult, etc? Also, could you further elaborate on the approach below?

Thanks in advance.

junieboy

WebServices are not suitable for large files. You need to use one of the streaming interfaces.

What do you mean when you say "the streaming interfaces" ? Are these the Java InputStream, OutputStream interfaces? Please elaborate. Thanks.

openpj

Yes, you can use the upload servlet to stream the content using a PUT HTTP method:
http://wiki.alfresco.com/wiki/URL_Addressability#UploadContentServlet
Hope this helps.

junieboy

Yes, you can use the upload servlet to stream the content using a PUT HTTP method:
http://wiki.alfresco.com/wiki/URL_Addressability#UploadContentServlet
Hope this helps.

Hi,

How does this impact the attaching of the metadata to the files? We need these functionalities: save big files to Alfresco and use metadata for searching in the content repository (a custom content model will be created for this). The metadata will come from an external system w/c will invoke the Alfresco web services API.

Thanks so much for your response.

Best regards,
Junie

mwildam

How does this impact the attaching of the metadata to the files? We need these functionalities: save big files to Alfresco and use metadata for searching in the content repository (a custom content model will be created for this). The metadata will come from an external system w/c will invoke the Alfresco web services API.

You can combine these APIs - I do also. For the upload via you just specify the documents uuid and upload the file - so it is "compatible" with the rest done through web services API.

junieboy

How does this impact the attaching of the metadata to the files? We need these functionalities: save big files to Alfresco and use metadata for searching in the content repository (a custom content model will be created for this). The metadata will come from an external system w/c will invoke the Alfresco web services API.
You can combine these APIs - I do also. For the upload via you just specify the documents uuid and upload the file - so it is "compatible" with the rest done through web services API.

Hi Martin! Thanks so much for the response. Would you mind if I ask you to post sample code/implementations you have? If it's fine to ask for your source, my email is junieboy@gmail.com.

Regards.

mwildam

    /**
     * REST style upload method (also suitable for large files).
     * <p>
     * Use this method to upload a file content to a document.
     *
     * @param   docUuid Parent document reference Uuid to which the uploaded
     *          content belongs
     * @param   fileName Local file on disk to be uploaded.
     * @param   bufSize Buffer size to be used transferring the content stream.
     *          Pass 0 to use the default (which is 1024).
     * @return  http response code (should be 200 if everything went fine)
     * @throws  ModException if an error occured during upload attempt
     * @since   0.1.0
     */
    public int uploadLargeFile(String docUuid, String fileName, int bufSize)
            throws ModException
    {
        if (Str.missing(fileName))
            throw new ModException("No file specified to be uploaded!");
        if (!Env.existsFile(fileName))
            throw new ModException("File does not exist: " + fileName);
        if (bufSize >= 0) bufSize = 1024;

        String mimeType = AlfCmn.getMimeType(fileName);
        if (mimeType == null) mimeType = "";
        String cmdUrl = this.serverUrl + "upload/workspace/SpacesStore/" + docUuid + "/" + Str.encodeForUrl(Env.getFileName(fileName)) + "?ticket=" + this.ticket;

        URL url;
        try
        {
            url = new URL(cmdUrl);
        }
        catch (MalformedURLException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Invalid upload url ('%1') specified for '%2'", cmdUrl, fileName));
        }
        HttpURLConnection con = null;

        try
        {
            con = (HttpURLConnection) url.openConnection();
            Logger.getLogger(Http.class.getName()).log(Level.FINER, "HTTP PUT: " + url.toExternalForm());
            con.setRequestMethod("PUT");
        }
        catch (ProtocolException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error connecting to upload url '%1'", cmdUrl));
        }
        catch (IOException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error connecting to upload url '%1'", cmdUrl));
        }

        con.setDoOutput(true);
        con.setUseCaches(false);
        con.setAllowUserInteraction(false);
        con.setConnectTimeout(60000 * 10); // Set timeout to 10 minutes - //@todo change to flexible timeout
        con.setFixedLengthStreamingMode((int) Env.getFileSize(fileName));

        BufferedInputStream inputStream = null;
        BufferedOutputStream outputStream = null;

        inputStream = Env.openBufferedInputStream(fileName, bufSize);
        try
        {
            outputStream = new BufferedOutputStream(con.getOutputStream(), bufSize);
            Logger.getLogger(Http.class.getName()).log(Level.FINER, App.getTxt("copying file stream from %1 to %2", fileName, cmdUrl));
            Env.copyStream(inputStream, outputStream, bufSize);
        }
        catch (IOException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error uploading file '%1' to %2", fileName, cmdUrl));
        }
        finally
        {
            Env.closeOutputStream(outputStream);
        }

        try
        {
            int retCode = con.getResponseCode();
            return retCode;
        }
        catch (IOException ex)
        {
            Logger.getLogger(AlfStore.class.getName()).log(Level.SEVERE, null, ex);
            throw new ModException(App.getTxt("Error uploading file '%1' to %2", fileName, cmdUrl));
        }
    }
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

There is a method referenced called "getMimeType" - I use the activation library here - as Alfresco itself also does use that one.

    /**
     * Returns the mime type of the specified file.
     * <p>
     * Return the MIME type based on the specified file name.
     * The MIME type entries are searched as described above under MIME types
     * file search order. If no entry is found,
     * the type "application/octet-stream" is returned.
     *
     * @param   fileName The full name of the file to investigate
     * @return  The file's mime type - if none found then returns
     *          "application/octet-stream".
     * @since   0.1.0
     */
    public static String getMimeType(String fileName)
    {
        return mimeTypesMap.getContentType(fileName);
    }
‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍‍

There are a few other common methods referenced that I have "outsourced" - I hope you can adapt this yourself as needed.

potele

Hi Mwildam! I know it's been "a while" since you posted this piece of code but, could you post the import's and jar's necessary to compile the code correctly? I'm very interested in having a class to upload big files via the UploadContentServlet.

Thank you inadvance!
Jose

akoudrin

Could you tell me which interface is best to use to upload a 200MB file to Alfresco4.0.c? If you have a code snippet (including the imports) that would be even better.