SMB2 / SMB3 server support

I saw that news over the weekend, and we have been discussing it is a team.

CIFS is a specific dialect of SMBv1, and so is likely to stop working when Microsoft disables their client support for it. Our implementation of CIFS is vulnerable to many of the security problems mentioned in the Microsoft blog post, and so for some time we have recommended people use WebDAV in environments where those risks are considered unacceptable.

As Axel Faust‌ has pointed out, Alfresco has not invested in improving our SMB support since Alfresco version 4.0. That work was to improve the performance and stability of the current implementation, and we have not done the work to support newer versions of that protocol.

When we have looked at upgrading to a newer version of the SMB protocol, it is an expensive undertaking. We have instead directed our resources to other areas, such as improving WebDAV and the new SharePoint support in the AOS library.

The news from Microsoft has forced us to re-evaluate our position with SMB/CIFS. There are some new libraries we could leverage for this support, but integrating them would still be a big project. I wonder if it isn't better to end-of-life our CIFS support and recommend WebDAV for all use cases required mounting Alfresco as a shared drive.

Here is my analysis of WebDAV versus SMBv3.1.1:

WebDAV is superior to modern versions of SMB when used for file access over a high-latency network. This is increasingly important as more deployments are in AWS or some other public cloud.
WebDAV has worse performance than SMBv3.1.1 on a local network.
WebDAV cannot handle transferring files larger than 4GB, so something like FTP would have to be used for those cases.

Am I missing anything?

Is anyone willing to share a use case shared drives where WebDAV would not be adequate?

All comments

For quite a while now Community members have urged Alfresco to update its SMB implementation. The issue REPO-1393 was filed by product manager Richard Esplin‌ to track the requirement. But there is no indication that this is something that will get any sort of priority from Alfresco, e.g. from the strategic vision / product roadmap presentations we had at BeeCon. Also the argument may be that even when Microsoft disabled SMBv1 by default, it will always be just a registry setting to re-enable it.

I saw that news over the weekend, and we have been discussing it is a team.

Here is my analysis of WebDAV versus SMBv3.1.1:

WebDAV is superior to modern versions of SMB when used for file access over a high-latency network. This is increasingly important as more deployments are in AWS or some other public cloud.
WebDAV has worse performance than SMBv3.1.1 on a local network.
WebDAV cannot handle transferring files larger than 4GB, so something like FTP would have to be used for those cases.

Am I missing anything?

Is anyone willing to share a use case shared drives where WebDAV would not be adequate?

dhant

Hello.

I apologize in advance if I am in something wrong. All statements are based on personal experience using WebDav.

At first glance, WebDav looks very attractive, but on closer inspection it turns out that this is not true.
The WebDav protocol has a number of drawbacks, compared to SMB:

Protocol overhead.
WebDav is implemented over HTTP. This means that each WebDav command will be passed to the server in the form of an http request. Thus, an http header will be added to each WebDav command. As a result, there is a situation when you need to transfer 100 bytes of http-header to transfer 10 bytes of the file. Slowdown is especially noticeable when copying a large number of small files. For this reason, WebDav disks are not suitable for synchronizing a large number of files (this may be the logs of some programs). Even commercial implementations of WebDav, such as "Synology NAS", can not cope with this task. From what I conclude that this is a protocol defect that can not be circumvented or compensated.
WebDav clients have different protocol implementations.
There is no strict compliance with the http://www.ietf.org/rfc/rfc4918.txt . Two WebDav clients can have great differences in the implementation of such things as: Preservation of cookies, sequence of execution of DAV commands, authorization method, supported encodings. Some WebDav clients may not support some of these features. I believe that support for the most common WebDav clients (built-in Windows client or davfs for Linux) is possible. For example, you can define the "dialect" of the WebDav client by the "User-Agent" line from the http-header. In this case, additional costs will be required to implement the "dialects" of the WebDav protocol for each specific client.
Weak support for the protocol on the part of Windows.
The native Windows tools for connecting WebDav drives leave much to be desired:
Restriction on the amount of data transferred - by default 50MB, through the registry can be increased to 4GB.

Basic authorization is disabled by default. You can enable it through the Windows registry.

The WebDav-drive is not connected directly as SMB, instead the "Webclient" service is used (based on a redirector kernel driver mrxdav.sys), which downloads the entire files to the temporary directory. Only after this, the file becomes readable/writable (The native NFS client works in a similar way). Even the cloud "OneDrive" is recommended to use through a special client, and not through WebDav.

By default, only tls 1.0 is supported. (Actual for Windows 7, the state of the newer OS is not known).

Thank you for the analysis Dilan Hant‌. This is exactly the sort of information I was looking for. It will be interesting to see what others think about this topic.

julian_cervino

In my experience CIFS is not a reliable protocol to use with Alfresco. Beside the complexity to clusterize that kind of access, we have found a lot of unexpected errors and performance overhead.

We have configured and recommended a solution for enterprise customers using a Linux box as a proxy. The Linux Box exposes folders using SMB to clients and mounts, using webdav, a shared folder in Alfresco. I know this pattern is not always applicable (because permission/user concerns) but it seems to work fine in our most common case (scaner devices linked to Alfresco).

We are studying right now with one of our customers the possibility of replacing the Linux box with a Windows Storage Server. This way we would have high-availability from end to end (right now the Linux box is a single point of failure).

My two cents.

I would assume a "dummy" proxy like you have described for your canner device use case would be unacceptable in almost all but the rarest use cases.

SMB 3.0 has a feature called transparent failover which would make SMB usable for clusterised Alfresco systems. The availability of such features in more modern version of the protocol is why some people in the community had already urged Alfresco to consider an update.

One option that was discussed in the IRC #alfresco channel was the potential option of using Samba and a Virtual File System backend to expose Alfresco without Alfresco having to deal with the intricacies of SMB themself and relying on a proven, continuously maintained industry solution.

iblanco

Although I quite agree with Julián I do even agree more with Dilan Hant.

Not being a fan of SMB protocol at all when I started working with Alfresco around 10 years ago I was very enthusiastic about the option of using webdav and forgetting about SMB.

But reality did bite me. As Dilan said each webdav client had a completely different behaviour. What worked right with davfs did not work on MacOS or with any other Linux implementation and viceversa. In the Windows scenario it was even worse, not only did the version of windows change the Webdav client that internally was used but installing Office might change the client.

In that sense the CIFS client in the windows machines was much less problematic. So usually my recomendation is:

- Use Share for your day to day work

- Use CIFS if you want to acces the files in a "file system" way.

- Use FTP for uploading batches of files (or the Bulk Import Tool if we are talking about really big loads)

The situation with the Webdav protocol might have improved during this years but it should be thoroughly rechecked before assuming that it is a good alternative to CIFS.

Richard, what about AOS? If I'm not getting it wrong I think that the "Sharepoint protocol" that AOS implements is really a derivative/modification/implementation of Webdav. Am I right ?

If so maybe this has the advantage of being more specific, is an specific implementation that works with an specific client so there is less variability and it might just work. Does this implementation somehow improve any of Dilan Hant's worries?

Anyone with better knowledge about AOS can comment on this?

iblanco

By the way, are Linux webdav clients able to connect to the AOS's "webdav"-ish endpoint?

The AOS "webdav" is providing a Microsoft-corrupted variant of WebDAV and is not fully RFC-compliant. It will likely work with clients other than WIndows / Office, but there is no guarantee that weird stuff won't happen if you use it on non-Windows systems.

vincent-kali

Some remarks:

- We're using CIFS intensively on most of our alfresco deployments (in a pure collaboration context, smb is used on local lan, and share interface is used remotely).

- SMB is preferred to WebDAV for performances, reliability and conflict management (in many scenarios it seems that webdav is not locking file when editing, as said above depending on webDAV client)

- SMB can be easily integrated into a DFS architecture which brings seamless file share solution and potentially High Availability.

- Alfresco is one of the rare ECM solution that is providing SMB support, and we love it !

Finally I do confirm that we love SMB support in Alfresco, and it would be a very bad news if this support was discontinued.

Vincent

Alfresco and CIFS/SMB are in a kind of love-hate relationship. At least speaking for our customers: 20 of 30 companies have choosen for Alfresco because of the CIFS implementation. On the other hand it is one of a few root sources of trouble (beside transformer framework, search, versions, retrying transaction handler, ...).

WebDAV: One of our biggest customers (>800 concurrent active users with connected drive in windows) had to switch to WebDAV/AOS since Alfresco's CIFS implementation doesn't support terminal services / SMB block tagging on the same session. It is working somehow but not perfect. There are lot's of bugs, limitations in the Windows and/or Alfresco WebDAV implementation (limitation on path length, supported characters, locking, strange behavior on delete, ...)

To make the answer short: WebDAV may be an replacement if you can live with all the limitations but scales better with hundreds or thousands of users. You may miss the active part of CIFS which is notifying a client for updates (which would make the protocol unusable on Macs: no F5 reload)

But no big change without a big chance:

One of the biggest pitfalls users face with Alfresco is the synchronous implementation of the fileserver implementation which is NOT compatible with the client OS concepts: client applications are used to delete and recreate files all the time, rename files before saving new versions etc. (well known from the shuffle mode concepts implemented in the fileserver / cifs implementation of Alfresco). I would vote for a new better implementation which works more like dropbox but in a manner of a connected drive: file actions are decoupled, recorded and consolidated based on some known events (e.g. on file close). In the mean time the file is saved on a local directory or on a server's temporary directory. We discussed this way internally years ago as an samba4 connector to replace Alfresco CIFS. This would be much more scalable and would additionally solve all the trouble outside of Alfresco. Alfresco should see only the real resulting new/changed document. Nothing else. Maybe we should start a fundraising now for this idea 😉

Troubleshooting WebDAV | Alfresco Documentation

The AOS endpoint is the recommended WebDAV endpoint on Windows systems, and should be more reliable on Windows than the generic WebDAV endpoint. It is a distinct implementation of WebDAV than the generic WebDAV that is provided by JLAN. But as Axel mentioned, it isn't standards compliant and so we are hesitant to recommend it for use on other systems. See:

Minor correction Richard: RFC-compliant WebDAV is not provided by JLAN. JLAN is only responsible for CIFS/SMB, FTP and NFS.

Thanks for sharing your experience and opinion. Internally, we have discussed whether the new (proprietary) Desktop Sync client isn't in fact a better way to go than the old-school Shared Drive concept.

It would be great to see someone implement a Samba 4 back-end for Alfresco proof-of-concept that we could use as a starting point for our efforts. Any volunteers?

For my personal opinion the new or old Desktop Sync does only address the road warrior requirements but not the office or remote power user: You always want to see all files without the need to replicate the whole enterprise on your desktop. Not to mention the requirement that you want to see edit metadata, call actions, etc.

What about referenced files like images embedded in a page or machine parts in a CAD file? This will never be handled by pure sync concept.

The closest solution / proof-of-concept I've seen was from a company located in Australia (I need to search in my brain to remember the name Edit: found it but unfortunatly the company seems to be dead: eoss-edrive | SourceForge.net 😞 They implemented an asynchronous, multicast capable alfresco file protocol service to be installed in two flavors: client runs on a client and/or on a (local) fileserver. The smart stuff was that they mixed dropbox and fileserver concepts:

In case someone "opens" the file for edit a lock token will send to the network (multicast) and/or to the server
If node is in the local cache the file will be openend from there otherwise from the server
every change on the file will be saved in the local cache only
in case the file change is final (e.g. the OS lock has been releases) the changes will be put on the network using broadcast in a secure way (similar to dropbox) and send asynchronously to the server
every client supporting the protocol will receive the change notifications and may get the updated bytes from the broadcast in the internal network - maybe without any interaction with Alfresco (I have seen this only from dropbox before)

Another also good approach: The CMIS-OS network drive connector (Adobe Drive) Thomas DeMeo and his team developed for the creative suite when he was product manager at Adobe:

locally seen as drive (OS part in C and SDK in Java)
content is only catched from the server when it is required so the drive acts similar to a browser (I'm unsure how the directory structure, metadata is saved/cached)
any temporary / not final IO (temp files , file shuffling new, locked files) will not be send to CMIS server
content will be cached locally and will be kept. This makes the implementation very fast for the user perspective because the OS works on a local file.
drive has concept / java API for context actions, sync, properties, extendable document property sheets, previews, ...

Adobe may not have the Marketing / customers / use cases to put this solution into the mass market. Maybe there is a chance you can get / acquire / license this solution before it is dying?

The samba4 binding would be a very scalable solution to be implemented in C and would be limited to linux platform (btw much better platform for Alfresco ...)

bgewehr

We tried a lot of solutions for enterprise content creation management. The need is not sufficiently fulfilled by oneDrive, box, dropbox or any other desktop sync tool. The need is to work together on thousands of files, not to take some of them with me. That's why the promising Alfresco approach of a single federated repository in a multisite enterprise scenario that delivers a simple network drive experience with a modern and social network styled web UI for search, collaborating and sharing was such a brilliant idea for our daily business. CIFS/SMB is key to that use case.

lascaux

I confirm that we are also using intensively CIFS successfully. Most of our users have no clue about the backend system behind the share drive... they just use it as other shares on their explorer.

it's fast, eays and meanful !

Losing CIFS would be a big issue for us !

Sylvain

talwid

Visuality Systems offers a commercial SMB stack that can be utilized here. The solution is portable and updated with the latests SMB3.1.1 dialect.

I bet as much as GPL3 is being marked as "nearly impossible to use" in the FAQ on their homepage, so could be the commercial license of NQE / NQ Storage for an open source product like Alfresco. On the other hand it would make it "easier" (argumentatively) for Alfresco to turn SMB support into an Enterprise-only, paid extension - which is what I gather so far might be the most likely of outcomes anyway...

angelborroy

Some weeks ago I installed an Alfresco Server for a factory. They had a production line with terminals based in Windows CE which only understood CIFS connections. Engineers designed assembling instructions in the office, produced a PDF and assemblers consumed them from production line terminal. In this case, CIFS is a must.

Besides the license discussion and which smb/cifs library to use I see a big challenge in separating alfresco jlan code from the desktop shuffle file / temp file mimic to support desktop apps behavior like ms-office or adobe apps with alfresco's node and metadata model. Today this is a huge unreadable bunch of code. With other words: where to plug in any other cifs library not to get all the problems again Alfresco had (and still has) in working on a mounted drive with desktop apps? Most dms vendors implement this kind of magic code on the client side but of course could be also implemented on server side if there's no client part.

shomeier

Hey Heiko,

I totally agree with you about the advantages of an Adobe Drive like CMIS connector. It's a pitty Adobe does no more support it. From a technical perspective it looked awesome: Java side was OSGi based if I remember right and it was extensible with own menus and even a complete new connector.

Though they do no more support it I hope Adobe someday and somehow provides their solution.

Anyway maybe with the available frameworks today it might no more be that difficult to implement an own Adobe Drive like solution:

- The most advancing task in my eyes is the native layer. With Java Native Runtime and some open source libraries it looks like there is no need to do anything native by ourself: One could implement a CMIS FUSE Filesystem this way.

- The next important thing would be the context menus in Mac Finder/Windows Explorer:

Liferay has build an awesome library which makes it possible to access the new FinderSync API via Java (called "liferay-nativity"). It provides implementations for Windows/Linux/Mac

Though there are for sure many other obstacles to overcome I think these two were the hardest. I could build a first runnable quick-and-dirty solution on Mac which mounts the Alfresco CMIS repository into a folder (lists folders, reads/opens files, lists versions in a context menu) in just some days (and I am a very slow programmer ).

And I saw another (experimental) implementation for Windows (leveraging Dokan) on Github.

Maybe this could be done in a community effort together with Alfresco?

Hi Sascha (I suppose you're shomeier@github),

your project sounds very cool and you should join with Alfresco Global Hackathon 😉

Unfotunately the dokan support in Windows is very limited and if you manage to setup a team you should take an eye on capsulate the required services and layers to get your work reusable and flexible. A colleague already created a proof of concept on linux 4 years ago (with no CMIS in mind): GitHub - LotharMaerkle/alffs: A FUSE filesystem for the Alfresco DMS repository server. Maybe you can get some insides from this code?

Personally I would prefer not to use CMIS if your intention is not really to have a connector for CMIS since it is more expensive (server load and implementation efforts) and unflexible. Instead I would prefer a custom REST-API focused on this use case.

Another point is that a DMS paradigm will never be fully compatible with a filesystem connector since any temporary files, deletes and recreates shouln't be seen on the Alfresco side. A separate project and/or Alfresco should take care about this with a separate layer which translates real, local OS operations into alfresco repo requests. Learned from the Alfresco CIFS shuffle code nightmare I think this should be moved on the client side but if you want to get more control over events like "when file is closed" fuse may be the wrong layer for that.

Good luck and keep us informed!

https://github.com/FrKaram/KS2.Drive

You can also find details about the Global, Virtual Hack-a-thon in this platform, e.g. projects and teams, as well as general information. (The page linked by Heiko also provides a link back to this platform)

shomeier

Hey Heiko, Axel,

yes it is that github repo. Sry I forgot to include a link (or maybe I just thought it is not worth to mention because actually the tough work is already done by the libraries "javafs" and "liferay-nativity". I just wired these libs together ).

The Hackathon sounds great but it is already next Friday. With my current projects I'm afraid it is too tight for me to get some free time.

I never tried on Windows myself. I just noticed that Dokan seems to have somehow revived in 2016.

I actually hoped I never have to dig into these deep layers with C but if so the code from Lothar Maerkle seems for sure helpful.

To be honest the idea actually was to implement a CMIS Connector but the topmost layer allows to mount any type of Filesystem: Any implementation of the interface java.nio.file.FileSystem can be mounted (as a reference I always use the Google InMemory Filesystem implementation for testing/developing). I really loved this concept in Adobe Drive.

So an implementation which triggers an own proprietary REST-API might also be possible (though, even on Java-Side it is still not a piece of cake imho).

With CMIS also many other parties could benefit. I am trying to reduce the server load by requesting only minimal information (filter attributes, not including relationships if not needed etc). Also I am experimenting with some client side caching.

I am not yet sure about what kind of problems might arise when implementing a writing capability (currently I mount a ReadOnly FS). Are you talking about some kind of "staging" layer like in git clients: "Pushing" to the server should only happen on manual user action? Hmmm...this actually might make sense if a user is working on a file/object which has references to other files/objects. A user might want to push all these edits in a single transaction.

vincent-kali

Hi all,

We're working on an AOS drive for Windows (based on WinFSP, much like Dokany) thats is supposed to be an alternative to SMB for Windows clients. As AOS is NOT based on JLAN server, we expect much better performances.

Suggestion or contribution are welcome: