I just got back to the UK from five days networking and presenting at The Open Group’s summer conference in Austin, Texas. It was a hot one – hot weather, hot food, and hot conversations. On Monday I gave a 45-minute talk entitled, “Making Standards Work – The ArchiMate Exchange Format in Use” and on Tuesday I was a member of the open panel discussing Open Standards, Open Source, and the concept of “Executable Standards”. More of the latter in another blog post, for this one I want to summarise my talk about the ArchiMate Exchange Format.
The main themes of the July conference were “Making Standards Work” and “Open Standards and Open Source”. The key message and theme of the conference was this:
We should be taking advantage of [Open Standards and Open Source] to support infrastructures that can enable the kind of Boundaryless Information Flow™ today’s digital enterprises need.
This event will focus on how organizations can use openness as an advantage and how the use of both open standards and open source can help enterprises support their digital business strategies.
When I first read this I knew what I needed to say in relation to ArchiMate and the ArchiMate Exchange Format. I took the words “open” and “boundaryless” as my two main riffs and spoke about how the exchange format has opened up the boundaries between tools and ArchiMate users.
Open
The big question posed by this conference was “What does it mean to be open?” Indeed. Well, let’s see what we have:
- Open Source
- Open Standards
- The Open Group
- Open – “Allowing access. Not closed or blocked.”
My job was to try and partly answer the more particular question – “What does it mean to be open in relation to ArchiMate?” Sure, ArchiMate is an Open Standard to a large degree, and there are those who might argue that more needs to be done in this area but, in my talk, I tried to show that the ArchiMate Exchange Format really has moved things on toward this “Boundaryless Information Flow”.
Having an Open Standard is the first thing that’s needed. When that happens, you can build interoperable tools. But tools need to make sure that they work together via that Open Standard. Consider web browsers and the old “browser wars”:
Indeed, they’re all great tools…
It’s the same in the ArchiMate world. There are some great ArchiMate modelling tools, but they need open standards in order to work. But here’s the thing – an open standard is not going to be interoperable unless there is an interoperable data format. Web browsers use HTML as their main interoperable language. In terms of interoperability, ArchiMate, up until recently, only existed on paper – a “paper tiger” as someone called it at the conference. Unless it is “executable” in terms of interoperability you’re just going to strengthen the silos in the tools world. This is not what we want.
Go back a couple of years. We find a variety of tools and services supporting the ArchiMate language; some are proprietary, some are open source, but all of these tools are storing their data in different formats, including Archi. Nothing wrong with that…
…unless you’re the customer.
Clearly, if you value your users and customers you need to satisfy the following:
- The possibility to use more than one ArchiMate toolset or provide options
- The certainty that one’s data will be accessible now and in the future
- Ensure one can access one’s data at all times without vendor lock-in
- Recognise that an open standard should support an open data format
This is where The Open Group’s ArchiMate Exchange Format fits in.
The ArchiMate Exchange Format
The ArchiMate Exchange Format (AEF) project got started in late 2012 and included contributors from major ArchiMate tools vendors (including myself representing Archi) and other experts. Over the following few years, many discussions and technical meetings were held to solve the various issues that arose. Some commentators felt that it was an impossible task, pointing to the problems in the UML and XMI world. My own take on this was that we needed to focus on providing a pragmatic solution and that we should try to remember what the great musician and guitarist, Robert Fripp, once said:
We begin with the possible, and move gradually towards the impossible
After a couple of false starts, the project made great headway during late 2014 and through most of the following year resulting in version 1.0 which was released in August 2015.
Essentially, the AEF is an XML data format that represents the core components of an ArchiMate model. It acts as a container for conveying the following things:
- Elements
- Relationships
- Properties
- Organizations (structure)
- Metadata
- Diagrams
Here’s an XML snippet:
The project’s statement of purpose made clear what the AEF is not:
- It is not the definitive representation (binding) of the ArchiMate language
- It is not a declaration of the ArchiMate metamodel (XMI)
- It is not intended to make demands that are not in the language
- It is not trying to change the language from outside in
And also what the AEF is:
- A pragmatic approach to exchanging ArchiMate models between tools
- Flexible
- Optional
- Accessible
- Straightforward to implement
In my presentation at Austin I gave a brief summary of the previous topics and then went on to ask the following question:
So, does it work?
To address that question I told two stories where we can see the AEF in use. The first story concerns the European Interoperability Reference Architecture.
The ArchiMate Exchange Format in Action – The European Interoperability Reference Architecture
The European Interoperability Reference Architecture (EIRA) is a reference architecture designed to guide public administrations in their work to provide interoperable European public services to businesses and citizens.
Here are the key phrases that you can find in their mission statement (emphasis is mine):
- “The need for interoperability in Europe is higher than ever”
- Building an “interoperable Europe”
- “Interoperable Software”
- “Agree on the language and the format to use to exchange information”
Sounds good. The project has chosen to use ArchiMate to model their architecture. Here’s a high level overview of the EIRA:
Archi has been used to create the ArchiMate models, diagrams and reports. It’s clear that they’re using an Open Standard together with an Open Source tool. As the Archi Product Manager I think this is great to hear but, as I pointed out in my presentation at Austin, this might not be enough.
Back in July 2015 John Gøtze reviewed and analysed the EIRA project for a Qualiware blog post. I think it’s a balanced and constructive piece. Further on in the review, John Gøtze makes this point:
The EIRA model is available as an Archi file. The data is also available in Archi-produced HTML and images.
From a maturity standpoint, this is only just acceptable. Even an Excel sheet version would be better, but better would be “raw data” available in a range of formats, possible as an api.
John’s point is important, and I agree with him. As I said in my talk in Austin, even an Archi file is “proprietary”. That’s right folks, “Archi Inc.” is a tools vendor just like the others, the only difference is that Archi is free and open source. As with the other tools, the data that Archi produces is not standardised.
Remember, this piece was written in July 2015. John goes on to note:
Of course, The Open Group is working on an ArchiMate Model Exchange File Format, and has sponsored the development of an Archi plugin for exporting to that format.
One month later, in August 2015, The Open Group released version 1.0 of the ArchiMate Exchange Format, with initial implementations coming from Archi (sponsored, as John notes, by The Open Group), Corso, and BiZZdesign.
Going back to the EIRA project website at this point in time we see this:
Here we can see that the ArchiMate model is available as an AEF file. Hopefully, this goes some way towards addressing John’s concern. But, for me, this may not go far enough, as we shall see.
You can see from the screenshot of the website that the EIRA project now has the ArchiMate model available as an AEF file. So, we have an open European project using an Open Source tool, Archi, with an Open Standard, ArchiMate, delivering an Open data format file. However, some of the other EIRA artefacts (the report and model diagrams) are still produced using a tool, in this case, Archi. So there’s still a dependency on a tool to generate the reports and diagrams.
Here’s the important part. The AEF is an open data format represented in a standalone XML file, validated by an XSD Schema, and hosted and maintained by The Open Group. It has now become an accepted standard (albeit in its early stages) that is fast being recognised as a key part of the whole ArchiMate project. Now, an ArchiMate model can exist outside of any given tool. Think about that. That’s an obvious thing, right? Obvious, yes. Simple, yes. And also profound.
The ArchiMate Exchange Format in Action – The Thorbjørn Story Arc
Here’s the second story I told in Austin. I call it “The Thorbjørn Story”. Actually, it’s a story arc that starts with Thorbjørn and looks set to run for a quite a few more episodes.
Thorbjørn Ellefsen is a cyber security expert who, back in 2015, was working for Difi, the Agency for Public Management and eGovernment in Norway. Thorbjørn is currently working for Capgemini and is a great advocate for Open Source, and has been using Archi and ArchiMate in his work, and sharing some of his ideas via GitHub. Actually, Thorbjørn spoke about all of this at The Open Group conference in London in April this year.
Some time in 2015, Thorbjørn highlighted the potential of using XSLT (Extensible Stylesheet Language Transformations) to work directly with an AEF file. As I said in a previous blog post, XSLT is a language for transforming XML documents into other documents or other formats such as HTML, plain text or PDF, and because an AEF file is in a straightforward XML format this is entirely possible. Thorbjørn did some initial experiments with XSLT and the AEF XML files and generated a basic HTML report displaying all of the concepts in an ArchiMate model. I was inspired by Thorbjørn’s work and took things a little further and managed to generate basic ArchiMate diagrams in SVG format. You can read about all of this in my blog post.
Here’s the key conclusion that I came to in that blog post:
But for me the important thing is that this clearly demonstrates that by using existing technologies based on XML, XSLT, HTML, and CSS one doesn’t have to resort to a proprietary tool in order to display and share that data. Both the data and the representations of that data can exist in an open, standalone format.
For me this is simple yet profound.
I also suggested that it should be possible to run with the ball even further, by using XSLT to generate full diagrams and representations of ArchiMate data in new and unexpected ways.
Guido Janssen
The next character in this story is Guido Janssen. Guido works as Lead Architect with PharmaPartners in the Netherlands. He contacted me earlier this year to say that he had indeed picked up the ball and was generating better SVG diagrams and adding additional functionality. Here’s a screenshot of how far he’s got:
Remember, all of this is generated from the original XML file in the ArchiMate Exchange Format, and then transformed by the XSLT to produce the output in HTML and SVG format. The diagram is rendered by SVG that’s taken from the source XML file. It’s a standalone process that you can do from the command line using open and standardised tools.
If you’re interested in following Guido’s great work he’s put the code on GitHub.
The Cathedral and the Bazaar
When addressing the big question, “What does it mean to be open?” I’m reminded of Wittgenstein when he said, “the meaning of a word is its use”. And the meaning of something like the ArchiMate Exchange Format is its use. Put another way, it’s as Eric S. Raymond proposed in his great essay, The Cathedral and the Bazaar:
Any tool should be useful in the expected way, but a truly great tool lends itself to uses you never expected.
The “tool” in our case is the ArchiMate Exchange Format. The “expected way” is that the AEF allows ArchiMate models to be usefully exchanged between software implementations. And the unexpected uses are just beginning to appear. I never expected the uses that I just described. And I never expected the next part of the story.
Synchronicity is a thing. This happens when the time is right and a notion occurs more than once crystalizing in a magical moment. With regard to the AEF this happened very recently with three occurrences of using the AEF data as a feed for graph databases (actually there may be a fourth occurrence, but I can’t find the reference on Twitter right now.)
Graph Databases
What’s a graph database? I’m no expert on the matter, even though I have a nascent interest in the subject, so I’ll let Wikipedia explain:
In computing, a graph database is a database that uses graph structures for semantic queries with nodes, edges and properties to represent and store data. A key concept of the system is the graph (or edge or relationship), which directly relates data items in the store. The relationships allow data in the store to be linked together directly, and in most cases retrieved with a single operation.
Now, clearly an ArchiMate model holds exactly this type of raw data with its nodes, edges and properties. Except that it doesn’t – if we can’t get at that raw data. Enter once again the ArchiMate Exchange Format.
Take a look at this post in the Archi User Forum. Poster “peterson.br” (presumably from Brazil) tells us what they used in the process:
- Archi (with OpenExchangeFile plugin)
- OrientDB (Graph and Document Database – http://orientdb.com)
- Java language
And what they did:
- Generated Java classes from OpenExchangeFile XSD (using JAXB)
- Exported .archimate file to OpenExchangeFile format (XML)
- Created a program in Java that imports the generated XML file and exports it to OrientDB
They took this:
And got this:
Now look at the next post in that same Archi User Forum thread. Here, Thomas Michem gives us a link to a blog post where he describes some similar work that he’s been doing with ArchiMate and graph databases. Here’s a summary:
At our recent Information Architecture event, we highlighted the important role of Information Modelling. One important aspect we emphasized was network or relation modelling. An upcoming trend is the usage of graph models. In this blogpost we will apply graph modelling to the Archimate modelling language. Archimate is used in Enterprise Architecture to model architectural elements (business processes, data objects, applications, etc) and the relations beween them. The classic way to explore an Archimate model is to create diagrams that show a specific perspective (‘viewpoint’). In this blogpost we show you how an Archimate model can be explored and queried as a graph.
In his blog post, Thomas walks us through the process that he used. He did the initial modelling in Archi, exported the model using the AEF and then used XSLT to get the data into another XML format, GraphML. Using Neo4J the ArchiMate model can then be queried as graph data.
It’s a great blog post and I don’t want to spoil the show, so I recommend you go and read it. To give you an idea of how cool all of this is, I’ll just leave this here:
Rackspace
Talking of cool, I met a couple of interesting guys at the Austin conference, Ben Truitt and Mark Morga, both working for Rackspace in Texas.
I spent a lot of quality time talking to Ben and Mark and they showed me some really cool things that they have recently been doing with Archi, graph databases and Cayley, an open source graph. The conversations that Ben, Mark, and I had (together with Ed Roberts of Elparazim) showed me that, once again, open initiatives such as Archi and the AEF really do open doors to innovation. Hopefully, we can continue working together in the coming months.
So, what happened? What next?
I’ve tried to describe here the key points that I made at the conference in Austin and I’ve tried to address the question, “what does it mean to be open?” in the context of ArchiMate, Open Source, Archi, and the ArchiMate Exchange Format. I said earlier that this is part of a bigger story arc, and that this is really the start of opening up the standard to a wider audience and opening the door to innovation. I’ve given a few examples of how a seemingly simple thing can lead to innovation in unexpected ways, and how the AEF is turning out to be more than the sum of its parts.
What’s interesting, for me at least, is not so much the AEF in itself but what arises out of this “bold endeavour”:
- Conversations and sharing of ideas
- Development of new ideas and technologies
- Innovation
- “Boundaryless Information Flow”
This is just the start of what’s possible and there’s more to do. We need to keep improving the exchange format, and come up with a new version that supports ArchiMate version 3.0. Also, ArchiMate tools may be required to implement the AEF in the future as part of The Open Group’s ArchiMate tools conformance process.
I’m upbeat about the whole thing, as are most of the people I talked to at the conference in Austin. The AEF provides a level playing field, solves the problem of interoperability and, I think, provides the missing piece of the puzzle for an open standard. And when I say “level playing field” I include Archi in that because, with the AEF, no particular tool is required to access, view and analyse the ArchiMate data, open source or otherwise. Using the AEF means that your data is open and standalone.
For me, all of this is exciting stuff. We began with what was possible, and by working together, we now find ourselves creating things that really do enable a “Boundaryless Information Flow”.
(Many thanks to Andrew Josey and the team at The Open Group for inviting me to the conference and sponsoring my visit)
