nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

As we transition to the new Hyland Community, some post links may be temporarily disrupted as we finalize redirects. We apologize for the inconvenience and are working to resolve this as quickly as possible. Details.

Document batch-upload

mikechenosky

Hi, does Alfresco support batch-upload of documents to its document library?
if so, how?
If not, can someone suggest how to get around it?
Thanks,
mike

Find more posts tagged with

Comments

mikeh

Hi,

FTP/CIFS/WEBDAV can all be used for batch update - also a user posted the code in these forums for a ZIP file import action where you upload a single ZIP containing all the files you want and it gets exploded out into the Alfresco repository.

Please see the wiki for CIFS and/or FTP configuration.

Thanks,
Mike

kevinr

Batch import via ZIP file upload has been added to the web-client UI for 2.1.

Thanks,

Kevin

gerald_quimpo

Hi Kevin, or anyone else,

Batch import via ZIP file upload has been added to the web-client UI for 2.1.

How do I do that (operationally, in the UI)? I've been looking around and it's not clear to me how to do it.

Gerald

tommorris

Off the top of my head, you can do this for WCM projects.
When you browse your sandbox for a particular web-project, the 'Create' icon drop-down list has a 'Bulk Import' option.
I think kevnr might mean that. Did you mean bulk import for regular DM spaces?

I did notice that the code for the regular ACP import action, can also import ZIP files…
So you can try a rule that kick starts that action for ZIP file-types.

Tom

gerald_quimpo

Did you mean bulk import for regular DM spaces?

yes, what I meant was bulk uploading many documents (in my prototype/test,
it's around 10,000 small (less than 512 bytes each) XML files. I don't need aspects,
title, description, etc to be set. I just need to push many files into the repository.

I find that writing files to alfresco via the CIFS or NFS share is very slow. In my
prototype/test, writing to a raw disk filesystem takes 7 seconds or so. Writing the
same files to CIFS takes 34 minutes or so. Deleting those files (or, actually,
the folder in which they reside and specifying all files and folders inside it to be
deleted too) is also very slow (I'm not timing it, but it's been around 3 minutes now,
while I've been writing/editing this post).

This is just my desktop development server, one drive, 2G of RAM, tomcat and
alfresco have 1G of that. but the overhead is very high. I'd be happy with writes,
reads and deletes being 5 times slower than to the raw filesystem. Even
10 times slower would be OK (my 10,000 files would be written in 70 sec).
But almost 300 times slower is a problem. Is this normal? Or are there well known
performance tuning tips that I might missed?

I did notice that the code for the regular ACP import action, can also import ZIP files…
So you can try a rule that kick starts that action for ZIP file-types.

Ok. I'll try to look into that. In fact, though, I'm eventually going to have to do this
in a program (users will upload a single large file, the program, or rule+action will
take that large file and split it into many individual records/xml documents). So
I'm looking at the web interface mainly as a proof of concept, that high speed
bulk uploading is possible at all. Then I'd wrap whatever the right technique is
in code.

Thanks for your reply!

Gerald

ttsherpa

Hi everyone,

I'm quite new to alfresco, but I have a testing installation that seems to work, and while doing a batch upload of several thousands very small text files into Alfresco Enterprise in one of our LInux servers I notice the extremely slow speed you mention (100 times slower than a raw file write!!!!). Is this normal? Did you find any solution or explanation to the problem? I use FTP but I guess that similar bad results were obtained with zip files in the web interface, or with cifs.

What do you think?

TIA

mdutoo

Hi

Don't be so sure, I didn't do a comparison on Alfresco 3, but on 2.x CIFS was notoriously faster than FTP…

lfcohen

We are also facing severe problems. We use WebDAV and need to migrate a lot's of files and folders, basically, a full structure into our Alfresco.

Not only the transfer is painfully slow even on the local network where the server is located, as well as doing it remotely.

And a major problem is when we try to send a structure with folders and nested folders and more nested folders, regardless if they have files or not, we get "internal server error 500" and major problems!

We are using both the native WebDAV "client" on Mac and Windows, as well as CrossFTP (using WebDEV protocol) on both platforms as well..

Any clue what might be doing this ?????

Thanks for any feedback!

Leonardo

mmurphy

I'm giving Alfresco a trial run and was hoping to give a recommendation to use if for our company's internal document management and collaboration. To say my experience so far has been "frustrating" would be an understatement. Anyway, I'm trucking along hoping it will all come together.

My problem as it relates to this thread is almost exactly the same as described above. FTP and CIFS are both very slow copying files. 200 MB took almost 30 minutes on a Gig Ethernet network. I tried copying the files to the server in windows then connected to the localhost share and copied the files that way into the repository. Local drive into the repository took about the same amount of time as over the network, close to 30 minutes.

This really isn't acceptable for a production environment. I have almost 2 TB of data to potentially upload into this repository, A task that will take about 2 and a half years at this rate.

mikeh

It's unlikely the network is ever going to be the limiting factor. What sort of server are you running Alfresco on? Also what database. Is the database local?

Alfresco scales very well, but you need to make sure you've got the hardware to match. http://www.alfresco.com/media/releases/2008/01/unisys-benchmark/

Did you see this wiki ariticle? http://wiki.alfresco.com/wiki/JVM_Tuning

Thanks,
Mike

sergisat

I have some problems too, I tested with Alfresco3Stable and with new Alfresco 3.2 in a vmware esx 3.5 host with 128GB RAM and the storage is stored in NetApp filer.

The virtual machine configuration is 4vcpu and 16Gb RAM with Centos 5.3 x86_64 / Ubuntu server 9.04 x86_64.
My database is Mysql 5.1.36 Community in another virtual machine with 4vcpu and 16Gb RAM in Centos 5.3 x86_64 .

My jvm configuration line is:
JAVA_OPTS="$JAVA_OPTS -XX:NewRatio=2 -server -Xcomp -Xbatch -Xss1024K -Xms2G -Xmx2G -XX:NewSize=1G -XX:+UseConcMarkSweepGC -XX:+CMSIncre
mentalMode -XX:MaxPermSize=256M -XX:CMSInitiatingOccupancyFraction=80"

Alfresco is slowly copying files over CIFS/FTP and more slowly deleting files over Alfresco Explorer.

mrogers

Is your alfresco running well in vmware?

Have you seen this thread http://forums.alfresco.com/en/viewtopic.php?f=8&t=19990&p=66193&hilit=vmware#p66193

Have you installed the Microsoft loopback adapter?

sergisat

I dont use Microsoft operating system for alfresco, I installed in Centos 5.3 and in Ubuntu Server 9.04.

Alfresco is working slowly in uploads/deleting files and if I explore nodes with 1500 elements is slowly too. I have folders in alfresco with 1500 elements and is more slowly to see. If folder have around 100 elements is more fast.

loftux

@sergisat
Maybe you can try to change the Batch size for indexing to defer the indexing. See http://wiki.alfresco.com/wiki/Full-Text_Search_Configuration
Are you storing the index on the NetApp? I do not know how fast a NetApp is compared to local disk, but the index storage location needs to be on a fast disk.
Have you looked at the mysql load? As you probably know

Alfresco also needs to do db updates to store a file, maybe you have the bottle-neck there? Or the communcation in between.
Your hardware for both Alfresco tomcat and the db server looks well sized, looks like it should be able to handle this load (such as indexing while saving).

usuf

I am having the same problem when uploading a directory with multiple files through webdav (painfully slow on a local Alf & local DB) . Has anyone found a way to speed this up?