Spring Roo, Skyway Builder and Code Generation
Last week at Spring One, Rod Johnson acknowledged in his keynote address what we here at Skyway Software have been saying for a long time: application frameworks combined with code generation technology are required to significantly boost Java developer productivity. It is great to hear SpringSource finally espousing the benefits of code generation and automating the “low value” coding items that developers oftentimes spend days, to even weeks, on while working on their projects. I have been touting these benefits for years, so it is great to see SpringSource finally recognizing the value.
Rod’s Ringing Endorsement of Code Generation was Much Needed
Roo is a great first start, but it has a long way to go to accomplish what we here at Skyway Software believe that enterprise java developers need in a code generator. There are many things a code generator should do, and we use several basic ones as our guide:
- Simple and intuitive input for the generator
- Open and extensible generator
- Generated code that looks “hand coded”
Generator Input
Skyway Builder is built as a rich set of Eclipse tooling. While Roo has you working from a command prompt to define the domain objects and controllers, Skyway Builder gives you a graphical view in the Eclipse navigator and within Eclipse editors to construct these artifacts, and more. We leverage all of the great things in Eclipse that developers are accustomed to using for constructing the generator inputs.
It goes beyond even the mechanism for developers to create an input though. What we often find is that organizations want to change the source of the input. Why should the input to the generator only be the style that we define? Isn’t UML a valid input style, or some custom XML structure that an organization may already be feeding to a proprietary system? Of course, the answer is “yes.” In Skyway Builder, you can point right to your existing database tables and generate scaffolding from them, or right click on a WSDL document and generate Spring services, or for those with an investment in UML, you can use your UML model as a starting point. This is also why we chose Eclipse as the foundation for Skyway Builder. One of the key projects in Eclipse is the Eclipse Modeling Framework (EMF). Even if you have never heard of EMF, if you are using Eclipse, you have used it. EMF drives pieces of many projects within Eclipse from Web Tools to Java Development. EMF provides us with the underpinnings to easily transform things like UML and XML structures into the Spring model that is fed into the generators. EMF is powerful, extensible and fast.
EMF is part of what has enabled us to support a diverse set of generator inputs. Through our partnership with IBM, you can create UML models in Rational that are stereotyped for Spring classes. There is also a custom transformation for UML to Spring, which enables you to then transform that UML model directly into Spring code. You can blend Spring stereotypes with plain Java class definitions all interrelated and within the same UML models. Coming this summer, you will also be able to reverse engineer pre-existing Spring applications into Skyway Builder and into UML. You can, of course, change either side — the code or the UML — through Skyway Builder editors and the changes take effect across the board. This functionality is something that large enterprises find very compelling.
UML is simply one well known example, but we have worked with other partners and clients to enable different XML structures as inputs to the generator as well. And while there are ways to accomplish similar things without EMF, why recreate the wheel? EMF gives us power and flexibility and a common way of doing things with which many Eclipse developers are familiar. This means that Eclipse developers will also be familiar with how to extend the generator.
Configurability & Extensibility of Generation
Different organizations will have very different needs for the output of the generator. Sometimes it will be basic things like file naming conventions or collection types, but some organizations will need to change the generated code itself. We see these items breaking down into:
- Configurations – simple settings that dramatically change the output structure of the generator
- Using “Impl” suffix for classes or “I” prefix for interfaces for example
- Picking collection interfaces and implementations; use of Set or List for definition and HashSet or ArrayList for instantiation for example
- Change packaging and locations of artifacts
- Extensibility – changing the processing of the code generator and the generated code
- Entirely change output of artifacts like controllers, services, configs, etc
- Add new artifacts for output; a Spring Batch file for example
These things are particularly important when augmenting an existing project. While the “green field” examples are fun, the reality is that the bulk of what goes on in enterprises is maintenance. The generator needs to be able to layer on top of what exists without breaking the design patterns previously used.
Extensibility is a critical component of a generator. What methods are generated? What does the code look like? How is it wired together? All these questions are critical, and businesses often need something tailored specifically for them. Roo talks about extensibility, but so far it is unclear how to extend or how much you can extend. As I mentioned in the above section on inputs, Skyway Builder makes heavy use of EMF and Eclipse extension points. Virtually everything Skyway Builder generates can be extended (or configured) and changed. If you have Eclipse development experience, you will find the extension of Skyway Builder particularly easy.
The Generated Code Matters … A Lot!
Having generated code that looks and acts in a way that the bulk of Java developers are use to is critical for a code generator. Code generators are wonderful, and save a ton of time. But ultimately, a developer will be in that generated code to manipulate it. Code generators just do not do 100% of the work. And if the code does not match traditional development patterns that your developers are familiar with, then the value of the code generator is greatly reduced. A good chunk of our partnership with Accenture has focused on this area in Skyway Builder. Accenture employs many 1000s of Java developers, so ensuring that the code looks and acts according to Spring and Accenture best practices — so that these 1000s of developers can instantly look at the code and know what they are seeing — is critical.
While Roo approaches code generation from a heavy domain model perspective (no service or repository classes and aspects on the domain object do all the work), Skyway takes a more common stance on code structure. Skyway Builder generates Controllers, Services, Repositories and Domain objects. I am personally not a fan of the heavy domain approach with the bulk of the code in an aspect attached to the domain object. I think aspects are great, where needed, but do not see the value in coding the service layer as an aspect on the domain rather than just coding the service. Most enterprise Spring implementations have complex business logic in the service layer, and often expose pieces of that logic as web services. This heavy domain model with aspects style is not something with which many Spring developers are familiar. This heavy domain object approach also runs into challenges with Spring Web Flow (SWF). Skyway Builder also happens to provide a beautiful and simplified SWF builder that works perfectly with generated services and repositories.
Summary
The bottom line is that I am thrilled that code generation is finally starting to be seen as the answer to eliminating Java complexity. Roo is a good first release for a code generator tool from SpringSource, but needs to grow in lots of ways to be what enterprises need for generation. Skyway Software has been doing enterprise class code generation for years, and if you find Roo interesting, then you should definitely give Skyway Builder a look.
Tags: Code Generation, Skyway Builder, spring, Spring Roo, UML to Spring
May 5th, 2009 at 5:20 pm
This is a very interesting blog post. I haven’t looked into extending Skyway Builder yet, but the fact that it’s built on Eclipse and EMF is very good for me. Most of the code generation I was considering customizing has already been done in 6.2.
May 6th, 2009 at 8:32 pm
I wanted to share some details of Spring Roo in light of specifics raised by Skyway’s blog entry.
First of all I’d to help readers of the Skyway blog be able to read about Roo for themselves, download a copy and give it a try. Skyway’s blog didn’t include a link, so I will do so here: http://www.springframework.org/roo and also http://blog.springsource.com/2009/05/01/roo-part-1/.
Moving onto the technical discussion, Roo does not use Eclipse EMF nor Eclipse technologies. Whilst SpringSource is fortuitous to employ some significantly talented Eclipse engineers (who lead popular technologies like SpringSource Tool Suite (STS), Spring IDE and AJDT), we carefully considered the various engineering issues and elected to build Roo outside Eclipse. The benefits of this decision include a <3 Mb download (not a 100+ Mb download!), compatibility with any IDE a person may wish to use (not just Eclipse), the ability to script the tool, the significant level of expressive conciseness and productivity possible through a text-based UI, a very lightweight and flexible add-on model, plus numerous other benefits.
It’s interesting that every modern mainstream RAD framework uses text-based UI paradigms (not IDEs like Eclipse). Of course, Roo has full Eclipse-based IDE support in the now-free STS product, enabling people who prefer to stay solely in their IDE to do so and still enjoy a truly first-class experience. The key point is Roo gives you the flexibility to use just the IDE, just the text-based UI, or both concurrently.
A more important point concerns the issues associated with modern code generators. I will write an entire blog entry on this topic soon, but in the meantime will simply observe that existing code generators generally have some limitations in the way that they operate. For example, existing code generators emit code in the same compilation units as they expect a user to maintain, or emit special “do not touch” compilation units that must be maintained by the code generator indefinitely. Some code generators prevent the user from editing Java whenever they want to, or need to specially mark certain members as “edited”. Other times code generators insist on being part of a build script, or require you to maintain special plugins within your IDE. Some have a questionable debugging story, limited SCM integration options, lack flexibility for significantly different architectural models plus facilitate an explosion of verbose, unnecessary and confusing layers that add little value. Existing code generators also generally have a limited to non-existent round-trip story, which is fundamentally critical to sustainable productivity, and some make it very unclear what is really going on. We took all of this (and more that I’ll blog about) into account when designing Roo, leading to a non-invasive, flexible and thoughtful model that uses aspects to ensure we can deliver meaningful compilation unit simplification whilst resolving the issues present in modern code generators. Aspect-based approaches underpin most modern Java technology (including Spring), so it’s hardly a big surprise that Roo would also use aspects given their repeatedly-proven mainstream usefulness.
The layering technique of a separate DAO/repository, services, domain and controller layer also deserves some reflection. Every modern mainstream RAD framework places persistence logic within the entity, which we believe is perfectly sound and pragmatic, especially in this era when JPA transparent persistence and object graph navigation undermines most of the initial motivations for dedicated persistence layers. In relation to services layers, these act as little more than transaction scripts in most practical usages. These transaction scripts delegate to so-called domain objects, which themselves are anaemic DTO-style objects that generally serve little architectural purpose beyond ORM and MVC form backing usage. This does not represent the maintainable, encapsulated, efficient OO we all should be striving to build, and is certainly inconsistent with the definition and motivation of the patterns purported to being observed.
For the sake of clarity, Roo does let you still write services layers. In fact if you are observing the actual architectural intention of services layers, they’re a very good idea. I wrote a services layer live on-stage last week at SpringOne using Roo, so it’s not particularly complicated – you just create a class as you always have. You can also write an add-on that emits a separate repository layer if you really believe you simply you must have one. Naturally Roo separates web controllers into their own layer by default. Whilst mentioning the web tier, I’ll also note there is no incompatibility with Spring Web Flow. The Spring project team works closely together and naturally ensure all projects work well together for the benefit of the community, just as we always have. Indeed Spring Web Flow functionality is going to be one of the next add-ons we provide out of the box.
Those concerned about layering should also take much comfort from the fact that Roo automatically delivers its users the SpringSource-considered best practice for modern enterprise applications built in the Java programming language. The people designing these best practices have been using Spring, helping the Spring community, professionally providing Spring services, speaking at Spring conferences and leading various Spring projects since as far back as 2003. Every engineering decision within Roo – and the architecture that Roo advocates – is underpinned by this significant history of pragmatism, community feedback, peer review and engineering integrity. We are pleased to debate in great depth any engineering decision associated with Roo.
In conclusion Roo brings the best of usability, productivity, flexibility and quality engineering within reach of enterprise Java developers everywhere. We solve the numerous problems of traditional code generation whilst also delivering SpringSource-considered best practice architecture. Given all of this is freely available in a lightweight open source project that also includes full IDE integration with the now-free STS product, we know the community is absolutely going to love it.
May 7th, 2009 at 3:39 pm
Ben, thanks for your post and comments.
My apologies for not posting the link to Roo — I should have thought of that, as I definitely think people should give Roo a look. Springfuse (http://www.springfuse.com) also makes a Java code generator for Spring, and this is another tool that I recommend Java Spring developers evaluate. If these tools do not satisfy what developers need for enterprise Java, my hope is that they will give Skyway Builder a try. Each of these tools seems to have a different driving philosophy, and there is plenty of room in the generation world for each.
Roo definitely has a lighter footprint than Skyway Builder and no prerequisites on Eclipse or EMF-based technologies. Later in this post, I will drill into some of the challenges that Roo users are likely to experience that we overcome by use of EMF. And while we have no text-based UI for Skyway Builder today, we will have one in our fall release. In fact, we plan to use the .roo syntax and scripting language for it, although users will still need to be in Eclipse or have Eclipse present on their machine. It would be great to collaborate (and involve others) around a common Spring scripting syntax for Java code generation.
I am not sure we are referring to the same things when we talk about running code generation in an IDE versus from command line. Just popping up the command line prompt in the IDE is not what I’m referring to when I say “running within an IDE.” With Skyway Builder, we focused on providing a natural experience in the IDE around keyboard shortcuts, integration into the environment, consistent UI, etc. All of this is done in Eclipse, which adds to our footprint, but we find these features compelling to developers who already spend most of their day in Eclipse.
Some of your assessments of “modern code generators” also seem to apply to Roo. Compilation units for the user and the generator definitely exist in Roo just as they do in other generators. One of our goals in Skyway Builder is to have as few of these as possible. For instance, it seems that manual changes or additions to some (all?) of the .aj files are overwritten the next time you issue a Roo command that manipulates that file. We have eliminated such limitations in Skyway Builder – thanks to that great EMF code from Eclipse. The same thing holds true for many of the utility files, such as web.xml, persistence.xml, Spring context files, etc. Roo does not need to change any of these config files today, since there is nothing to amend to controllers and no services. But as you broaden out Roo to encompass services, web services, URL aliasing, Groovy integration, UML, multiple databases, or other more complex generation items such as Skyway Builder enables today, you will find a need to either enable complex file merging or have generator owned files. Developers manually edit such files on a regular basis, even when using a code generator, so failure to maintain those changes when the generator is applied again is a big problem that all generators need to overcome. Skyway Builder handles just about all of these things today in ways that give enterprise developers the flexibility they need to generate some things and hand code others.
The Domain Driven Design (DDD) approach for Roo is definitely interesting. The Java community seems pretty split on the issue with some loving it and some hating it. From an industrialized Java perspective, DDD has not yet hit mainstream. Of the handful of excellent Spring books that I use and that most of the world uses, there are almost no examples of Spring best practices using DDD. Actually, aside from Roo, I couldn’t find any other Spring examples that use this DDD approach. Is Spring changing their best practices approach for Java from their Repository/Service model to a DDD model? If so, this would be great to know. If this is the new approach, we would be happy to support that style in Skyway Builder as well. We are completely agnostic and want to support whatever implementation style that mainstream developers like best.
I completely disagree with your categorization of services in that they “act as little more than transaction scripts in most practical usage.” Most enterprise applications make use of complex business logic within services and often expose some of those services as web services. Sure, operations in a service often function as transactional blocks as well, but “little more than transaction scripts” is hardly a fair assessment of how developers use services today.
Jack Kennedy, our VP of Engineering, has been working on a piece outlining a more philosophical perspective on code generation. Look for Jack to post that blog here in the next week or so. We would be very interested in your opinion on the topic as well.
May 7th, 2009 at 10:54 pm
Thanks Jared for your thoughts.
There are a multiplicity of techniques available to address the sort of problems that both Roo and Skyway aim to solve. I find it generally useful to categorise these techniques into whether they apply at development time (like Roo, Skyway, Eclipse gestures), build time (like JSR 269, XDoclet) and of course runtime (like LTW, dynamic proxies, DSLs). We prototyped most of these before we settled on the Roo approach, as I will explore in a future blog.
Whilst EMF is a good technology and we were fully aware of it during prototyping, we decided to use a different approach. We don’t believe the use of EMF or lack thereof constitutes a fundamentally important issue in either direction. I’d also add that in my experience most people aren’t particularly interested in the details about which technologies a tool internally uses to operate. Instead most people care about what the tool does for them and how its usage patterns and style affect the way they naturally work. I’ve had very few people ask me about Roo’s metadata models, add-on structure, process management infrastructure, abstract syntax tree operation, project model abstraction interfaces, shell parser and binder APIs etc.
I just wanted to clarify that Roo routinely round-trips files for which there is a robust DOM or AST model available. For example, take the “install security” command. This command goes in and edits web.xml, adding the required filter, filter mapping and application context information to the correct locations. We use editable file models extensively where they make sense.
We made a very conscious decision around the location of Java members that will be subject to automatic round-trip maintenance. We place such members by default in .aj files instead of .java files because we strongly believe this is the best place for them. The alternative of simply adding members into .java files via AST would have been easy for us to do, as illustrated by our existing commands that modify .java files via AST (eg add field). Indeed the user can even move members into .java files and Roo will work, and we’re already working on automating this (mostly so people can see how easy it is to permanently remove Roo from their project). Having said that, we strongly recommend against cluttering .java source files with both hand-written code and generated code. That’s what most Eclipse users do routinely with commands like generate getters/setters, hashCode/equals and toString, and then users have to routinely delete them all to re-run the command when things change (and have some process for establishing whether they changed any generated logic etc). It’s an unpleasant approach and Roo unashamedly avoids it. Our preferred model of compilation unit separation also means we need not maintain metadata in terms of which members a generator produced as opposed to the user wrote or modified, improving robustness, future add-on compatibility and further reducing clutter. Put differently, we have what we consider to be very sound reasons for compilation unit separation from a user productivity perspective. It’s certainly nothing to do with Roo engineering limitations, as our existing AST usage visibly demonstrates.
I see you completely disagree with my remarks about service layers. These remarks are based on my professional experience design reviewing real-world applications over many years. Most developers call something a “service layer” and “domain layer” when they’ve actually built a “transaction script” that uses “anemic domain objects”. I’ll defer to Fowler for a more thorough treatment of this problem, which he too has observed: http://martinfowler.com/bliki/AnemicDomainModel.html. Service layers have their place, but you shouldn’t need code generators to build them if you have a quality domain model in place. Whilst Roo was not designed with DDD in mind, the natural consequence of using Roo is a more object oriented and properly engineered system. Both of these are goals of DDD, although Roo does not have any preference as to what modelling techniques you use. One of the nice things about Roo is you can go back at any time and change something and it will still work. There’s no central UML model or up-front “design” needed – people can just build an app incrementally and iteratively.
At a higher level it’s not really the internal technologies we build on that matter. The differences that matter are how you use the tool and how adopting the tool affects the way you build enterprise Java applications on Spring. Skyway adopts a very graphical usability paradigm that appeals to a certain group of people, whereas Roo aims to deliver quality tooling to the programmer who is comfortable with Java concepts and simply wants a significant and sustainable productivity boost whilst using Java. Both tools therefore have a very different approach to initially building an application, and certainly how you maintain them long-term. Some people will feel more naturally comfortable in one tool or the other, and that’s perfectly fine.
I’m sure over time we’ll see more innovation cross-pollination between different tools, such as your proposal to implement scripting support based on Roo commands. Even with such innovation I am sure the core usability and engineering concepts central to the design of each product will remain inherent.
May 8th, 2009 at 9:16 am
Users of code generators definitely do not care about the implementing technology – at least until they run into a wall where the generator does not fit their architecture, process or inputs. At that point, extensibility and configurability suddenly become critical. As the generator targets more complex scenarios that are often boilerplate (some of which I listed above), we found great value in leveraging Eclipse.
I definitely understand your need to have Roo-owned compilation units – it is pretty common in generators. Our philosophies simply differ here. We see the blending of generated code and hand code as the norm in the software lifecycle at most enterprises, which led to our focus on having a very small set of such owned units. In fact, I do not think we have any that are actual code, just a handful of configuration files.
As Fowler points out, the style of DDD is definitely gaining steam, but today, DDD does seem to still be in the minority and many Spring shops currently put logic in the service layer. You may think this means they have a poor design because they would not need this logic in the service if they used a DDD approach. You might also be right. But, that does not change the fact that lots of shops have logic in the service that is far more than a transactional template. It also does not mean that developers are going to instantly change to DDD.
As I mentioned above, other than Roo, Spring is not yet pushing the DDD pattern in their examples. There are few references to DDD in the most commonly used Spring books, and I have only seen a few enterprise implementation success stories. There are also many, many Spring developers that do not use the DDD style. I try to take a pragmatic approach on such debates – since there are often pros and cons to most architectural styles, the problem being addressed often dictates the best style. At Skyway, we strive to make a generator that fits the most commonly used patterns. As user demand for DDD grows, or when the Spring community supports it as the recommended best practice, we will be happy to support DDD just as we do other styles. It would be great to know Spring’s official stance on the topic.
As an FYI, a DSL is often a development-time implementation. Both the Roo and Skyway Builder metadata models and the syntax driving them could be classified as DSL. There is no hard and fast rule that a DSL is runtime only, and the trend is actually moving towards development-time DSL.
Users will inevitably pick the code generation tool that best meets their needs. The premise of collaborating on the .roo script is to give developers flexibility and portability. Having a common script seems something with few cons that could greatly benefit the community in the end.
May 25th, 2009 at 12:29 pm
Neither or you really address why you would prefer design time “code generation” instead of more contemporary runtime approaches like DSLs that have become more mainstream in the Java community with the advent of Grails. Take for instance GORM (which now can be isolated from Grails to use it standalone); why not replicate what is done with Groovy, as it is Grail’s DSL, in Java through the use of annotations or a rich DSL. This seems like it would it would remove the traditional limitation of existing code generators with “don’t touch” sections and so forth, not to mention the amount of code that is typically generated and that would ultimately need to be maintained. I would be interested to hear your opinions on this.
May 27th, 2009 at 1:58 pm
Good questions I’d like to see discussed as well, Luke.
July 28th, 2009 at 7:19 pm
Luke, GORM works essentially by using Java reflection.
I have personally found two main disadvantages to this approach
1. The dynamically generated methods are not there at development time. This makes it hard (impossible?) to take advantage of compile time type safety – one of the features that makes Java often a better choice than Groovy/Ruby/Python for large projects.
2. Performance. The spring guys stated that one of their goals was for Roo not to degrade performance, extensive use of the reflection api rules out the GORM approach.
November 2nd, 2009 at 7:02 am
[...] Spring Roo vs Skyway Builder Tags: Java, Roo, Spring, Web Development Share this post! Twitter Digg Facebook Delicious StumbleUpon Google Bookmarks [...]