Top 10 Principles for Software Generation
I thought it might be worth while to start to document Skyway’s point of view regarding enterprise software generation, and the things that we think are important in this space. Our goal is to promote an active conversation and to solicit feedback of all sorts to ensure that we are working to address those areas that the industry feels are most important to the adoption of generative techniques and ultimately the simplification of delivering enterprise software.
Our Top 10 Guiding Principles for Software Generation
- Offer Incremental and Additive Approaches – Developers should be able to realize the benefits of software generation on their own terms using as much or as little of a generation system as they wish
- Extensibility is Key – The generation system must be extendible, configurable, and customizable at every level
- Exploit Every Asset - The possible sources of inputs to the generation should be as diverse as possible and be expandable by developers
- Support Every Developer – Provide tooling options that enable the widest set of development approaches and experience levels
- UML
- Skyway DSL / Visual Editors
- Other DSMLs (either UML-based or MOF/EMF-based)
- Java
- Command Line Interface
- Don’t Repeat ANYONE – Enforce the DRY principle by acknowledging that Java code is not the only place, and often is not the first place, where the definition of software concepts may occur.
- Offer Options for Post-Generation Changes – Changes to generated artifacts should be supported through a combination of:
- Source Round Tripping
- Intelligent Merging
- User Editable Dependency Injections
- Generation Componentization (Smallest possible components)
- Avoid Lock-In – Generated artifacts should be indistinguishable from those that would be written by hand by a developer following best practices. Developers should be able to walk away from the Generation approach at any time.
- Technology Changes as Often as the Requirements – Provide a clean translation from conceptual design to implementation remaining as permissive as possible to long term syntactical changes in the implementation language, architectural approach, and runtime framework
- Generate on Demand, Automate by Request – Give developers control over when and how the generation and reverse generation processes will run (Automatically or “On Demand”)
- Ex Uno Plures (Out of One, Many) – Deliver as much functionality as desired with as few inputs as required
In my next few blogs I’ll begin to add detail to these concepts and dive into the Skyway implementation a bit, where we are, and where we are going. I’d love to hear more feedback from the community about what principles you think are important. I’d also like to know what things you currently spend time on that you would like to see automated or generated in the future. In the interim, you can check out our open source project here.
UPDATE – The second installment in this series of blog posts has been published here.
Tags: Code Generation, Eclipse EMF, Enterprise Software Generation, Guiding Principles, java, spring, uml
June 17th, 2009 at 2:32 pm
To your #4, I would add DSMLs (either UML-based or MOF/EMF-based.
June 17th, 2009 at 2:58 pm
Thanks Charles, totally agreed.
In my next set of posts I will detail a bit more about the Skyway implementation, but our DSL is actually implemented in EMF and our “Visual Editors” are designed to make editing that EMF based DSL easier.
When I listed “Visual Editors” in the list, it assumed a prior knowledge of Skyway’s Visual Editors, and our DSL implementation.
Thanks for helping to drive towards clarity on that subject, I will update #4.
Jack
June 18th, 2009 at 5:35 pm
As a former user of a code generation tool, I think that point number 8 is very important. The input for the code generation should be completely abstracted from the implementation target and not trying to just increase the abstraction level from the implementation target. Thuse the tool I used had data manipulation concepts that were different from SQL, even if most of the targetted databases were be relational databases.
June 19th, 2009 at 8:38 am
Thanks Franco,
We definitely agree.
The approach we took was to basically create a layered architecture to defining the DSL where each layer or extension to the tool could “decorate” the lower layers. For example, when it comes to modeling your Domain objects, there are obviously a lot of options to capture a data model and many existing sources of truth. For some, the definitions of the Domain Model originate in UML and work down towards the persistence layer. For other developers, or other projects, they may be starting with an existing database or XSD schema. Some simply start with a POJO. So Skyway implemented its DSL to abstract what is common about all of these formats at the lowest layer, the definition of Domain Objects have fields and relationships, and then added extensions to allow for the later configuration or decoration of the Persistence information, or the JAXB information, etc. Those who wish to extend Skyway can layer in their own meta data and add their own modified code and configuration templates in order to capture information that they need to automate additional code generation beyond the current POJO, JPA Entity type generation that ships with Skyway.
Thanks for posting your experiences with generation.
We would love to hear more.
Jack
June 24th, 2009 at 9:24 am
[...] « Top 10 Principles for Software Generation Jun 24thSkyway’s “Perspective” on Software Generation jkennedy | Uncategorized [...]
June 24th, 2009 at 9:50 am
The second installment to this blog post can be found at http://www.skywayperspectives.org/blog/?p=709.
June 24th, 2009 at 7:59 pm
[...] can help code generation succeed? Says Skyway’s Jack Kennedy, code generation tools must offer options for post-generation changes. They must support round tripping, intelligent [...]
June 26th, 2009 at 12:28 am
I’ve found, In general, that the best application of code generation is for plumbing, i.e. getting your pure business code into/deployed/integrated with a framework/platform/system, etc. Anything that does not represent business logic should be generated. As an example, I’ve use code generation for decorating pure Java code for EJB 2 as well as Webmethods deployment (at that time using xdoclet).
It keeps the framework specifics out of my Java code. This kind of pattern is very useful to abstract out the specifics of the framework/deployment/platform plumbing and allow for development of the core business logic with only Java developers and do not need framework/deployment/platform specialist knowledge and skill across the whole team.
The naming must abstract enough that it does leak in any framework/deployment/platform specific grammar into your code.
That’s one of my annoyances, albeit very minor, with some of the new Java Annotations. The packaging and naming of them are leaking framework grammar into our code, i.e. javax.ejb.persistence.Entity. This implicitly leaks into the code grammatically that the POJO is a ejb3 entity. They should have kept the namespace and naming agnostic to something like java.lang.annotations.Persistable, then it would not have leaked into POJO and let the deployment environment determine the persistable framework, i.e. ejb3 if it was deployed into an ejb3 container or JDO or SQLMap, etc.
July 9th, 2009 at 10:27 am
[...] as #2 in Jack’s “Top 10 Principles of Code Generation” blog post, extensibility is key. We believe that a good code generation system must be [...]
July 9th, 2009 at 11:04 am
[...] The full blog post can be found here:http://www.skywayperspectives.org/blog/?p=688 [...]
August 6th, 2009 at 10:17 am
This post recently received the following comment on a separate discussion forum. For completeness of this thread, the comment and response are below.
Comment from Ben Gillis:
> #7, Generated artifacts should be indistinguishable from those that would be written by hand
Not sure I follow this one.
My auto-gen’d code goes in a separate file with a header over it saying it is auto-gen’d, “do not edit” warning.
Rule: no mixing of auto-gen’d code and manual code in the same file.
August 6th, 2009 at 10:22 am
Thanks Ben,
Jack’s point about avoiding lock-in was not so much about mixing hand code and generated code, but about the flexibility of allowing developers to understand and maintain the generated code if needed. This primarily encompasses how the code looks, where the code is generated, and scenarios including the absence of the code generator during enhancements or maintenance.
You are correct, best practices dictate generated code should contain such syntax and reside in separate folders (and Skyway is no different). However, there are valid use-cases for developers to blend hand code and generated code. For example, code generators that only generate stubs. Another example we have seen is in team development where you have more experienced team developers working with less experienced developers. The overall concept of avoiding lock-in is to allow this flexibility so that any developer in any scenario can use the code after it is generated.
August 6th, 2009 at 12:14 pm
Good clarification on #7. Generated code should look the same as hand-built code because development, debugging, and just plain figuring out how the code works often requires going through generated code and framework code, that’s one reason why open source frameworks are so useful. Making the generated code correct per corporate standards should be as easy as post-processing through jalopy or various other utilities. Adding additional post-generation processes should be easy to do, using widely available tools and frameworks on top of a standard build tool or framework.
All generated code should also be free of all compiler warnings and errors as reported by common utilities such as FindBugs, CheckStyle, and PMD. The entire starting project (i.e. maven archetype) should be easily created and should generate and compile and run and pass all generated tests immediately out of the box. Unit and integration tests should be generated from the input model and contain reasonable defaults for all data values (i.e. using mock objects) while still being easily modifiable. There should be a very clear separation of artifacts that are generated every time from those that are generated once (stubbed out implementations), in order to make updates and synchronizations between multiple users within a source code control system as easy as possible.
August 13th, 2009 at 8:25 am
[...] system must be extendible, configurable, and customizable at every level” (read our full list of code generation principles). We recognize customization as a fundamental requirement, and this, we think, is one of the [...]
November 18th, 2009 at 3:57 am
hi jack, you remember me the “essentials” from jack kerouack, i will like to talk to you about skyway builder and .net (i have some generators for .net open source), please email me to talk about.
p.d.: there its a code generation comunity in http://modeldrivensoftware.net/
thanks for alll