ldap authentication et.al.
Hello everybody,
is there an existing implementation to connect YAWL to an ldap-server (namely M$-ADS) for authentication and extracting of org-data (roles,capabilites,org-groups,organizatoins) ?
Any help is appreciated.
Thanks in advance
Roman
Hi Michael,
thanks for pointing the direction.
Cheers, Roman
I just discovered the LDAPSource class in svn repository form 5/03/2010.
It seems to do what we want and is configurable via a .ini file.
If I understand it correctly, the ini file has to be in this folder:
\engine\apache-tomcat-6.0.18\webapps\resourceService\WEB-INF\classes\org\yawlfoundation\yawl\resourcing\datastore\orgdata\
Is this right? I think that's very confusing. Why not store the config files in a more central place?
Also the sample ini file says:
# the name of the Role attribute name.
roles=o
In our case we have "memberOf" attributes, so I would just put
roles=memberOf
in, right?
Thanks!
neo
I tried to use the LDAPSource class and technically it works - if our repository had only like 100 users. But for 30000 users it fails completely because this is the MaxPageSize limit of the ldap servers here. And I had a hard time degugging it because there are no log messages created in the LDAPSource class. I added these and finally caught an exception which was just swallowed before. Why?? It didn't appear in the console or the logs!
Exception:
javax.naming.SizeLimitExceededException: [LDAP: error code 4 - Sizelimit Exceeded]; remaining name 'ou=users,dc=...'
This gets thrown at getNameList() in LDAPSource while iterating with "list.hasMore()".
Now my question: Why is this LDAP query - fetching all users - actually executed and not lazy-loaded? In our case we just need associations to Roles. I know this is a different approach but in a real world you just can't fetch all users and display them in a listbox or anything.
Do you have any idea how this could be done for the moment until a real solution is found?
Kind Regards
neo
Hi neo,
Thanks for your feedback. I've amended the LDAPSource class so that you can now set a max size limit (in the properties file) and it will read the entries in chunks of that size. I've also added some logging and more commenting. Thirdly, at your suggestion, I've moved the properties file to the same place as the other properties files (ie. webapps/resourceService/WEB-INF/classes).
While it all works with my testbed LDAP server, it doesn't have a large number of users so I'd be grateful if you could give it a test and let me know how it goes (the updates are in the svn repository now).
Thanks, Michael.
Hi Michael,
thanks for working on this. At the moment I can't test it because I'm outside of the domain environment.
But I still have concerns. Even if this solution technically works how would you really work with these imported organizational data in the editor and web interface? All the input elements are basically select boxes displaying every user and every group, in my case roughly 30000.
In my opinion there has to be a fundamental change in resource management for YAWL. Maybe I can illustrate it a bit more. Look at these two interfaces for resources:
http://activiti.org/javadocs/org/activiti/IdentityService.html (newly born BPMS)
http://yawl.svn.sourceforge.net/viewvc/yawl/trunk/release2/src/org/yawlf...
http://yawl.svn.sourceforge.net/viewvc/yawl/trunk/release2/src/org/yawlf...
I think the following methods should be completely removed and replaced by something else which scales well:
getParticipants()
getRoles()
getParticipantMap()
getRoleMap()
getParticipantCount()
getParticipantNames()
getRoleNames()
....
You see where I'm getting at... Methods trying to get ALL information will sooner or later fail... your workaround for LDAP is ok for the moment but it will probably fail anyway because all data is held in memory and this has a limit too.
I think it's ok to import data from the external data store into the yawl db but only on-demand and not in full.
I'd be happy to discuss this further with you.
Kind Regards
neo
Hi neo,
Thanks for your input - i fully appreciate where you are coming from.
The YAWL resource perspective's model is finely grained, in that tasks may be assigned to individual participants (as well as roles). This leads to two basic requirements (of course there are many others): the ability for designers to choose individual participants for task assignment (in the editor), and maintaining (and persisting) a set of work queues for any individual participant who currently has tasks assigned to them.
A decision was taken early on, in consultation with industry partners, that given the individual participant-level support YAWL provides, the best solution was to load the organisation data model on startup (and optionally refresh the data at regular intervals). This means designers can make choices from a list, rather than have to know and search for usernames etc. over and over again. Also, it allows work items to be allocated to queues without having to continually lookup whether the participants specified are valid. There are many other sound reasons for doing things this way. Believe me, this design decision was carefully considered and only agreed to after all bases had been covered.
The other consideration was to provide a means to load organisational data from any data source. YAWL provides the interface to do just that, so that the internal database can be used, or if desired any kind of external data source format (database systems, ldap, XML, spreadsheets, plain text, etc. etc.) can be 'plugged in'. Therefore, the YAWL org data model has to be able to handle these different formats in a generic fashion. If YAWL was designed purely for ldap, and nothing but ldap, then the model could be tuned specifically to it, but flexibility requires that some small tradeoffs are justified.
Having said that, YAWL is not unusual in the pre-loading of org data. Specifically, it is not unusual to load a subset of data held in an ldap directory in this way. The fix implemented in response to your original message is not a workaround, but a solution, using standard javax.naming interfaces and methods. It was a simple oversight on my part to not take into consideration the server side limits set on the number of entities returned per search request, but reading ldap data in chunks it the standard and recommended way of doing things.
A populated Participant record consumes around 150 bytes on average, so even with 30000 users that equates to 4.5Mb of memory consumed, which really isn't that much, especially after taking into consideration the amount of hardware that would be available to support 30000 users in any case.
The implemented org data model works well for all data sources, but of course we will continue to provide improvements to it as required.
Cheers, Michael.
Hi Michael,
Thank you very much for this extensive explanation!
I think I can understand the reasons now for the interface and loading of org data into yawl on startup.
I still think that for the visual interface it's not the right way to display every participant in a select box. It's ok for a small number of people but for like 30000 it's just not very comfortable to scroll in it etc. and when you are e.g. delegating a task to another user in the webinterface the browser will just slow down on slower pcs if it needs to load and display every person. I'm not yet sure how one could do it in a better way while not loosing the comfort for a smaller number of people.
What do you think?
neo
Hi Neo,
I think you are right, building lists of 30000 participants to choose from is not an efficient way to handle things (this is of course a separate issue to the internal loading of org data on startup).
Maybe a way to deal with it is to provide a (configurable) upper limit on building full user lists, after which some other form of search or lookup kicks in (e.g. search on userid or name or role etc, with wildcards accepted). It will take a bit of work, though.
I'll let the idea coalesce for a while - if anyone has any ideas in this regard I'd appreciate hearing them.
Cheers, Michael.
Hi Roman,
No LDAP implementation currently exists, but the YAWL resource service offers a 'pluggable' interface for organisational data sources. Briefly, each YAWL org data source extends from the abstract class org.yawlfoundation.yawl.resourcing.datastore.orgdata.DataSource, which has a small number of methods to be implemented for the particular data source - basically the methods map data from the source into the objects YAWL recognises. There are two implementations currently in the package: HibernateImpl (the default) and jdbcImpl (a vanilla jdbc implementation) - these can be used as examples on how to go about mapping an LDAP source.
Cheers, Michael.