nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

As we transition to the new Hyland Community, some post links may be temporarily disrupted as we finalize redirects. We apologize for the inconvenience and are working to resolve this as quickly as possible. Details.

Contributing functionality

ronald_van_kuij

Hola,

With years of detailed experience in jBPM, including the BPMN parts of jBPM4, I made the switch to using Activiti. In the company I currently work for, I convinced people to do a proof-of-concept with Activiti. The most important initial tests will be 'crash restistency' regarding transactions in the db, including crashing the db etc… Since I'm convinced this will succeed (maybe after some small patches), I'm already focussing on additional functionality that we need. Some of it is on the roadmap and I hope/think we can contribute in realizing this earlier or offloading some work for Joram as it seems (see below) or maybe he can focus on more complex things like boundary events (like ACT-15 we want those as well ;-)) and Tom on ACT-32

Foreach: parallel execution based on variable collection or expression-return-value collection (5.2, Joram).
I've already implemented something for this (only for tasks). It might not be how you should do it, since I just needed something to work for demonstrations, but I'm happy to make the required changes

EJB invoke (5.3,Joram)
The same as for the previous item

Asynchronous continuations ACT-126 (5.5, Tom)
The same as for the previous to items 🙂

JobExecutor
In addition to the previous requirements, I've made some changes to the way the JobExecutor works. This improves the troughput of the Async jobs by 30%, because of a lot less queries to the db if used in a high volume STP kind of way. What effectively happens is that jobs are put in the db, directly pinned to the server they are created on and put in the threadpool (if there is room). This way job acquisition is normally not needed for async functionality and only if the queue was full. Still then, retrieving them is not in a competing way since they are already pinned. Not even when failover is taking place since we changed that way failover works as well. Servers are registerthemselves in the db and update a record there notifying that they are alive. The combination of changes decreases possible issues when running multiple job executors on different machines since jobs are pinned in advance (I have not seen any problems at all!!!, so ACT-234 is probably not an issue anymore

We also added the option to have multiple queues so you can e.g. have a high-priority queue and a low priority one (or one per service). Pausing/Resuming is also added and behaviour to back-off if the queue becomes full. Configuring queue sizes, threads etc per queue is on the todo list, as well as extending Activiti Probe (if this is required).

I'm curious though what the ideas are behindACT-34, Refactor JobExecutor threading. We might take that into account if needed.

There still is the option though to have the current behaviour. It should only be made confirurable in the config (but I've learned those internals as well so that will not be a problem)

Expressions in timer duration ACT-429
Since I'm an expert on this from the jBPM era, I'm happy to contribute here as well since we need it to. No work has been done on it it yet (and certainly no documentation

)

'Dynamic Subproceses'
jBPM had the option to decide which subprocess you were going to call based on the value of process value. This is very interesting for us to. I've not realy seen a jira issue for this. The ones I did saw were about runtime created subprocesses, but that is not what we require. Thoughts?

Ok, this was the first post, expect more…

Find more posts tagged with

Comments

tombaeyens

What a start, Ronald!

Foreach: Joram plans to do that one this release.

EJB and JMS activities: would be great if you could contribute those.

Async continuations: I think I need to do that myself as there is already a lot of undocumented design ideas half in the code for that.

What effectively happens is that jobs are put in the db, directly pinned to the server they are created on and put in the threadpool (if there is room).

GREAT IDEAS! Now why didn't I think of that?! 😉 Kudos! You're very welcome to implement this, but do consider what kind of testing you would write for that. ACT-34 would be great if you could take that in one go. Currently we manage the threads ourselves. This should be done with a pluggable thread pool. There some JSR that specifies for appserver environments how you can obtain a threadpool that is managed by the server. (as you're not allowed to manage threads on your own in j2ee environments). Most j2ee servers don't check that by default, but I heard that websphere does.
Pluggability threadpool pluggability should be created in such a way that we don't introduce a dependency on any library when you use our own thread pool.
Another thing related to the JobExecutor is that some of the tests are failing once and a while on windows. We didn't have time yet to figure out if it is the test or the implementation that is the problem.
Also expanding test coverage on the job executor would be good. Especially high throughput or high load simulations would be great to have.
In our opinion, refactoring of the JobExecutor would be the most important and urgent. Would it be ok for you to start with that one?

Expressions in timer duration ACT-429: yep, assign to you if you want. good addition.
Same for dynamic subprocesses issue. Would be a great issue for you to do.

ronald_van_kuij

Foreach: Joram plans to do that one this release.

ok, since one part that I already did was not my satisfaction, I'll wait what Joram comes up with. Keep in mind though that loop and serial 'foreach' are easy but the parallel one might require a kind of async functionality if it needs to be limited in one way or another or it might even be more difficukt if all run in one transaction. Not sure if it is technically related to async but it mit be

EJB and JMS activities: would be great if you could contribute those.

I will. I remember the discussion from a while ago (pre Activiti) on how these should be configured. Are those still valid? I'll ask my collegue to implement the shorthand version and do the long bpmn one myself.

Async continuations: I think I need to do that myself as there is already a lot of undocumented design ideas half in the code for that.

The jobhandler functionality was clear, so I made an AsyncJobHandler. Qurious what the rest is, since my implementation is not that complicated at all. Since it scheduled for 5.5 I'd be happy to still help out with you giving some pointers and do the validating. I am already fairly familiar with the internal workings of Activiti 😉

What effectively happens is that jobs are put in the db, directly pinned to the server they are created on and put in the threadpool (if there is room).

GREAT IDEAS! Now why didn't I think of that?! 😉 Kudos! You're very welcome to implement this, but do consider what kind of testing you would write for that.

Well, I just implemented it in Activiti. The ideas are from a collegue of mine, Frederik Wiers

ACT-34 would be great if you could take that in one go.

will give tat a try

Another thing related to the JobExecutor is that some of the tests are failing once and a while on windows. We didn't have time yet to figure out if it is the test or the implementation that is the problem.

I run everything on Linux, but Frederik is still on windows. So hopefully it does not show up anymore with the changes, but if it does we'll check it out

Also expanding test coverage on the job executor would be good. Especially high throughput or high load simulations would be great to have.
In our opinion, refactoring of the JobExecutor would be the most important and urgent. Would it be ok for you to start with that one?

sure, since we want/need to use a JBoss threadpool for other reasons as well.

ronald_van_kuij

What a start, Ronald!
Also expanding test coverage on the job executor would be good. Especially high throughput or high load simulations would be great to have.

I used the tweet test to produce messages with unique content. This way it was easy to test if duplicates would be created or messages missed. What went wrong is that hitting the connection limit of the database, the (multi-threaded) creation of messages would fail and the test would sometimes stop. So some changes need to be made there but when I configured things in a way the connection limit was not reached, I could get a fairly constant rate of 420-460 jobs per second produced AND consumed on a 2GB Intel Centrino2 laptop where MySQL was running on that same laptop. (h2 in mem mode was 6000)

Coming week, I'll try to run a test with Activiti instances with an adapted JobExecutor (still with it's own threads) on Linux and Windows against the same remote db and see what happens. We'll also be crashing the DB at that time to see how the jobexecutor (and later on Activiti as a whole) keeps behaving. This area is where there has been bad experiences with JBoss JMS and which lead to a custom queueing solution… Activiti has to be om par here what was custom developed. 🙂 But comparing the custom queue and the jobexecutor gave me the feeling that we could merge some functionality and 'switch'

We'll keep you guys posted.

bernd_ruecker

Wow, cool. Especially the JobExecutor stuff seems reasonable and makes a lot of sense. It is very good that you start to focus on these enterprise kind of things. Cool!

jbarrez

Awesome! Welcome back, Ronald!

Offloading work from me is always something which I like 😉

ronald_van_kuij

Awesome! Welcome back, Ronald!

Offloading work from me is always something which I like 😉

Offloading? No way… you'll get other thing to do, no worries mate…

sangv

Ronald -

I know I am reading this a little late, but in the first post here, you had mentioned about dynamically running subprocesses based on a process value and that there was no JIRA issues for that. We needed the same functionality and this issue was created by one of my teammates some time ago

http://jira.codehaus.org/browse/ACT-831

Can you take a look at this and see of this is what you meant. Also, can we have this feature so that we don't have to hard code the sub process key…

Thanks,
Sang

ronald_van_kuij

I'll have a look, but important for me to check the correctness of te patch is to have testcases. So if you can provide some, that would make the testing easier. And the next step ten is to have some documentation as well.

sangv

Sounds good. I will move my teammates patch to 5.9-SNAPSHOT, add test cases and create another patch.

Thanks,
Sang

sangv

Hi Ronald -

Provided an updated patch, test cases and documentation for http://jira.codehaus.org/browse/ACT-831. Please take a look and let me know if you need any other information.

Thanks,
Sang

ronald_van_kuij

Thanks, will look into it tomorrow or thursday

sangv

Hi - Did you get a chance to look at the updated patch for this issue. Please let me know if you have any questions.

Thanks,
Sang