These are the news items I've curated in my monitoring of the API space that have some relevance to the API definition conversation and I wanted to include in my research. I'm using all of these links to better understand how the space is testing their APIs, going beyond just monitoring and understand the details of each request and response.05 Jul 2017
I saw a blog post come across my feeds from the analysis and visualizaiton API provider Qlik, about their Qlik Sense API Insights. It is a pretty interesting approach to trying visualize the change log and road map for an API. I like it because it is an analysis and visualization API provider who has used their own platform to help visualize the evolution of their API.
I find the visualization for Qlik Sense API Insights to be a little busy, and not as interactive as I’d like to see it be, but I like where they are headed. It tries to capture a ton of data, showing the road map and changes across multiple versions of sixteen APIs, something that can’t be easy to wrap your head around, let alone capture in a single visualization. I really like the direction they are going with this, even though it doesn’t fully bring it home for me.
Qlik Sense API Insights is the first approach I’ve seen like this to attempt to try and quantify the API road map and change log–it makes sense that it is something being done by a visualization platform provider. With a little usage and user experience (UX) love I think the concept of analysis, visualizaitons, and hopefully insights around the road map, change log, and even open issues and status could be significantly improved upon. I could see something like this expand and begin to provide an interesting view into the forever changing world of APIs, and keep consumers better informed, and in sync with what is going on.
In a world where many API providers still do not even share a road map or change log I’m always looking for examples of providers going the extra mile to provide more details, especially if they are innovating thike Qlik is with visualizations. I see a lot of conversations about how to version an API, but very few conversations about how to communicate each version of your API. It is something I’d like to keep evangelizing, helping API providers understand they should at least be offering the essentials like a roadmap, issues, change log, and status page, but the possibility for innovation and pushing the conversation forward is within reach too!
I have been learning more about Apicurio, which is the open source API design editor I have been waiting for. There are a number of things I’m interested in when it comes to Apicurio, but one side element that caught my attention was their road map.
I am a big fan of encouraging folks to share their roadmap. It is an important part of helping establish a shared future between API provider and API consumer. Apicurio is an API tool, without any APIs (yet), but the roadmap purpose remains the same. I like how Apicurio shares their tech preview, beta, and 1.x plan, in a coherent and organized way–you do not have to be a developer to understand what they are planning.
As I was using Apicurio I had a lot of questions about what it didn’t do. I had a lot of ideas about what it should do, and before I set out writing these ideas on my blog I spent some time with their road map, syncing items on my list with items on their roadmap. After I was done, I had reduced my list of questions and ideas to just a handful of items–which I will write about shortly. Their proactive, coherent, and complete road map saved me time, but also will save them time when it comes to listening to my feedback (if they do).
Complete, coherent, and plain English road maps are another one of those super simple, captain obvious ideas that over 50% of the APIs I review DO NOT HAVE. Which is why I’m writing about it and showcasing a positive example like Apicurio. Please do not forget to share road map with your community–it will save us both time.
I am stepping back to today and thinking about a pretty long list of API design considerations for the Human Services Data API (HSDA), providing guidance for municipal 211 who are implementing an API. I’m making simple API design decisions from how I define query parameters all the way to hypermedia decisions for the version 2.0 of the HSDA API.
There are a ton of things I want to do with this API design. I really want folks involved with municipal 211 operations to be adopting it, helping ensure their operations are interoperable, and I can help incentivize developers to build some interesting applications. As I think through the laundry list of things I want, I keep coming back to my audience of practitioners, you know the people on the ground with 211 operations that I want to adopt an API way of doing things.
My target audience isn’t steeped in API. They are regular folks trying to get things done on a daily basis. This move from v1.0 to v.1 is incremental. It is not about anything big. Primarily this move was to make sure the API reflected 100% of the surface area of the Human Services Data Specification (HSDS), keeping in sync with the schema’s move from v1.0 to v1.1, and not much more. I need to onboard folks with the concept of HSDS, and what access looks like using the API–I do not have much more bandwidth to do much else.
I want to avoid moving too fast for my API audience. I can see version 2,3, even 4 in my head as the API architect and designer, but am I think of me, or my potential consumers? I’m going to seize this opportunity to educate my target audience about APIs using the road map for the API specification. I have a huge amount of education of 211 operators, as well as the existing vendors who sell them software when it comes to APIs. This stepping back has pulled the reigns in on my API design strategy, encouraging me to keep things as simple as possible for this iteration, and helping me think more thoughtfully about what I will be releasing as part of each incremental, as well as major releases the future will hold.
I have had a number of requests from folks lately to write more about Github, and how they can use the social coding platform as part of their API operations. As I work with more companies outside of the startup echo chamber on their API strategies I am encountering more groups that aren't Github fluent and could use some help getting started. It has also been a while since I've thought deeply about how API providers should be using Github so it will allow me to craft some fresh content on the subject.
Github As Your Technical Social Network
Think of Github as a more technical version of Facebook, but instead of the social interactions being centered around wall posts, news links, photos, and videos, it is focused on engagement with repositories. A repository is basically a file folder that you can make public or private, and put anything you want into it. While code is the most common thing put into Github repositories, they often contain data file, presentations, and other content, providing a beneficial way to manage many aspects of API operations.
The Github Basics
When putting Github to use as part of your API operations, start small. Get your profile setup, define your organization, and begin using it to manage documentation or other simple areas of your operations--until you get the hang of it. Set aside any pre-conceived notions about Github being about code, and focus on the handful of services it offers to enable your API operations.
- Users - Just like other online services, Github has the notion of a user, where you provide a photo, description, and other relevant details about yourself. Avoid making a user accounts for your API, making sure you show the humans involved in API operations. It does make sense to have a testing, or other generic platform Github users accounts, but make sure your API team each have their own user profile, providing a snapshot of everyone involved.
- Organizations - You can use Github organizations to group your API operations under a single umbrella. Each organization has a name, logo, and description, and then you can add specific users as collaborators, and build your team under a single organization. Start with a single repository for your entire API operations, then you can consider the additional organization to further organize your efforts such as partner programs, or other aspects of internal API operations.
- Repositories - A repository is just a folder. You can create a repository, and replicate (check out) a repository using the Github desktop client, and manage its content locally, and commit changes back to Github whenever you are ready. Repositories are designed for collaborative, version controlled engagements, allowing for many people to work together, while still providing centralized governance and control by the designated gatekeeper for whatever project being managed via a repository--the most common usage is for managing open source software.
- Topics - Recently Github added the ability to label your repositories using what they call topics. Topics are used as part of Github discovery, allowing users to search using common topics, as well as searching for users, organizations, and repositories by keyword. Github Topics is providing another way for developers to find interesting APIs using search, browsing, and Github trends.
- Gists - A GitHub service for managing code snippets that allow them to be embedded in other websites, documentation -- great for use in blog posts, and communication around API operations.
- Pages - Use Github Pages for your project websites. It is the quickest way to stand up a web page to host API documentation, code samples, or the entire portal for your API effort.
- API - Everything on the Github platform is available through the Github API. Making all aspects of your API operations available via an API, which is the way it should be.
Managing API Operations With Github
There are a handful of ways I encourage API providers to consider using Github as part of their operations. I prefer to use Github for all aspects of API operations, but not every organization is ready for that--I encourage you to focus in these areas when you are just getting going:
- Developer Portal - You can use Github Pages to host your API developer portal--I recommend taking a look at my minimum viable API portal definition to see an example of this in action.
- Documentation - Whether as part of the entire portal or just as a single repository, it is common for API providers to publish API documentation to Github. Using solutions like ReDoc, it is easy to make your API documentation look good, while also easily keeping them up to date.
- Code Samples w/ Gists - It is easy to manage all samples for an API using Github Gists, allowing them to be embedded in the documentation, and other communication and storytelling conducted as part of platform operations.
- Software Development Kits (SDK) Repositories - If you are providing complete SDKs for API integrations in a variety of languages you should be using Github to manage their existence, allowing API consumers to fork and integrate as they need, while also staying in tune with changes.
- OpenAPI Management - Publish your APIs.json or OpenAPI definition to Github, allowing the YAML or JSON to be versioned, and managed in a collaborate environment where API consumers can fork and integrate into their own operations.
- Issues - Use Github issues for managing the conversation around integration and operational issues.
- Road Map - Also use Github Issues to help aggregate, collaborate, and evolve the road map for API operations, encouraging consumers to be involved.
- Change Log - When anything on the roadmap is achieved flag it for inclusion in the change log, providing a list of changes to the platform that API consumers can use as a reference.
Github is essential to API operations. There is no requirement for Github users to possess developer skills. Many types of users put Github to use in managing the technical aspects of projects to take advantage of the network effect, as well as the version control and collaboration introduced by the social platform. It's common for non-technical folks to be intimidated by Github, ad developers often encourage this, but in reality, Github is as easy to use as any other social network--it just takes some time to get used to and familiar it.
If you have questions about how to use Github, feel free to reach out. I'm happy to focus on specific uses of Github for API operations in more detail. I have numerous examples of how it can be used, I just need to know where I should be focusing next. Remember, there are no stupid questions. I am an advocate for everyone taking advantage of Github and I fully understand that it can be difficult to understand how it works when you are just getting going.
Many of the core areas of my API research, and the common building blocks of the API life cycle that I talk about regularly, often seem trivial to the technically inclined, or the purely business focused segments of my audience. To many, having a road map might be a thing you have when developing and deploying an API, but really doesn't matter if you share publicly. I'd say, with technical folks they often don't even think of it, with the more business focused individuals often deliberately choose not to, seeing it as giving away too much information to your competition.
I think Slack nails the reason why you want to open up and share your API road map. I can talk about this kind of stuff until I'm blue in the face, but people just don't listen -- they need leadership like Slack brings to the table. In their post today, they open with:
We know that being a developer is hard, and building on a platform is not a decision to be made lightly. Many platforms have burned developers and we frequently see that risk highlighted. This is our response.
They nail the the (often elusive) promise of the API ecosystem:
An ecosystem, a real platform, is shared. We are growing fast, but no one company alone could grow fast enough to meet the amount of potential in front of us. We are working hard, but no one team can work hard enough to meet the demand that lies before us all. Instead, we are building a platform where this potential, this demand, is shared. As we grow, developers are able to succeed with us; sharing our customers and joining us in changing the way people do work. In turn, our customers are delighted, new customers have even more reason to use Slack, and the cycle continues.
They nail the reality of the API ecosystem, and how it is a shared experience:
An ecosystem in its healthiest form creates a virtuous cycle. Platforms do fail. We’ll make mistakes, but we’re building for something much greater. We are building for a future where Slack is dwarfed by the aggregate value of the companies built on top of it. This is our success as a platformâ—âwhen the value of the businesses built on top of us is, in sum, larger than we can ever be.
They connect the reality of an ecosystem with having a public road map:
So, today we’re sharing our platform product roadmap. It is a small step in equipping you to claim this opportunity with us. There are three major platform product themes to highlight: app discovery, interactivity and developer experienceâ—âyou can see more on this card.
Slack doesn't stop there, and throw in having a idea showcase to help as well:
We’re also sharing what we’re calling an Ideaboard —âa list of useful ideas that could be built into Slack apps per conversations we’ve had with our customers. [...] The goal of the Ideaboard is to continue this momentum by creating a bridge between our developer community and our customers’ needs.
As Slack mentions, this won't be perfect. They don't have all the answers. A public road map and idea board might not ultimately make or break Slack as a platform, or ensure the community continues to evolve as a full blown ecosystem. Shit can go wrong at any point, but having a public road map, and active, truly contributing idea board, will go a long way to contributing to positive outcomes.
If you aren't allowing input from your API community into your road map, truly considering this input as you craft your road map, and then share the resulting plan with your community, how can you ever expect them to stay in sync? It doesn't mean that Slack will listen to everything, and every developers opinion will be included in the road map. It will however, put Slack into a more open state of mind, and go a long way to set the right tone in the API ecosystem, building trust with the individuals and business who are building on their platform.
What tone has been set in the community around the APIs you provide? What is the tone in the communities you exist in as an API consumer?
Each of the Adopta.Agency project runs as a Github repository, with the main project site running using Github Pages. There are many reasons I do this, but one of the primary ones is that it provides me with most of what I need to provide support for the project.
Jekyll running on Github pages gives me the ability to have a blog, and manage the pages for the project, which is central to support. Next I use Github Issues for everything else. If anyone needs to ask a question, make a contribution, or report a bug, they can do so using Github Issues for the repository.
I even drive the project road map, and change log using Github Issues. If I tag something as roadmap, it shows up on the road map, and if I tag something as changelog, and it is a closed issue, it will show up on the change log--this is the feedback loop that will help me move the clinical trials API forward.
A road map for your API, is one of those essential building blocks that can go a long way in building trust with your API consumers. Sharing your plans, helps developers prepare for the future, and better plan for their own road maps, keeping everything potentially in sync.
From my vantage point, a simple, up to date, easily found road map is a building block that benefits both API provider and consumer. Road maps help provider organize their plans, and figure out how they are going to communicate it with the ecosystem, then setting the overall tone for how API consumers will engaging with a platform--both good and bad.
During my regular monitoring of the space, I saw that Microsoft released a pretty slick "Cloud Platform Roadmap", which is yet another sign for me on how Microsoft is stepping up their cloud (API) game. Their interactive road map allows you browse the roadmap by areas like cloud infrastructure or Internet of Things (IoT), and see features that are recently available, public preview, in development, cancelled, or have been archived.
Beyond the core functionality of the road map, I'm impressed that they published a blog post telling why they did it, acknowledging that it will bring visibility into platform operations. Additionally there is a prominent button, providing an email address where you can send your suggestions for the road map, providing evidence of how a roadmap can contribute to the overall feedback loop for an API platform.
Overall, I like the simplicity of the Microsoft cloud platform road map, and it is definitely a healthy building block I'd encourage other API providers to consider offering as well. I'm going to make sure Microsoft's roadmap is one of the examples I cite along the API management line, as part of my overall API life cycle map.
As I work to create Swagger API definitions for the almost 1000 companies in my API stack, I'm chasing an elusive definition of a complete Swagger definition for any API. Only a small portion of APIs I am defining using Swagger will reach an 80-90% completions status, in reality most will never be complete for reasons that are totally out of my control.
When it comes to crafting API definitions for public APIs, I may be able to get the API definition complete in the sense that it has all the paths, parameters, responses, definitions, and security configurations, but if the API itself is missing pieces--the API definition will always be incomplete (in perpetuity).
As I work to automate the questions I ask of Swagger definitions, helping me get to a more coherent definition of what is a complete Swagger definition, I can't help feel the need to offer API design tips to the API providers I am profiling. Honestly I have never considered myself a high quality API designer, I've been more business than tech, but I can't help pointing out the obvious stuff.
I am still using my API definition tooling for my own goals around defining the API Stack, but because I'm building it all as APIs, I intend to open it up to others for analyzing their own API definitions, determining how complete they are, and potentially offering up tips on what could be done to improve them. A kind of self-service, API design tip API, that helps with some of the most common elements that are missing from API designs.
In 2015, API definitions are opening up a buffet of services that are courting API providers ranging from mocking and deployment, to monitoring and security. I don't see why API definitions couldn't also allow for professional API designers to help API providers evolve their APIs in a more constructive ways, from the outside-in. Using Swagger or API Blueprint, professional designers could easily provide suggestions for existing API providers and their designs, then give them potential suggestions that they could use in their API design road-map.
Anyhoo, food for thought!
I needed a place to keep track of all the things I will be working on around the FAFSA API project. To help me keep track as well as communicate to you, I've added a roadmap page to this project.
If you have anything you'd like to see get done, you can submit as an issue and I'll either add as milestone or find other way to make sure the work gets done.
A roadmap is an essential part of a healthy API ecosystem. The transparency and communication that come with providing a roadmap for your API and open data initiative will go a long way in building trust with your community.
The City of Philadelphia is sharing its roadmap of data sets that it intends to open in the coming months, by providing a release calendar using cloud project management service Trello. In addition to seeing the datasets the city intends to release, you can also see any data sets that have already been published.
According to the open data roadmap, Philadelphia is releasing data on street closures, energy consumption, evacuation routes, campaign finance, bike racks, budgets, expenditures and city employee salaries to name just a few.
This type of transparency doesn’t just build trust with citizens and developers, it provides an incentive for the city government to deliver high quality public data, in a meaningful and timely manner. There is still a lot of work to be done once this data is available, in order to develop quality analysis, visualizations, other APIs and tools that can be used in mobile and web applications. But what Philly is doing is a great start.
The City of Philadelphia’s approach to its open data roadmap is good to see. It is something all cities should have, as well as all API owners. Your roadmap will set expectations within your community, guide your own initiatives and provide a healthy blueprint for moving your efforts forward.
When you operate an API, you need to make sure and communicate with your developer community about what your plans are for the future, so developers can plan their own roadmap, and keep in sync with the coming changes to the API.
In addition to issue management using Github, API providers can use Github to manage their roadmap using the milestones features of the social coding platform. Milestones can be directly related to issues or can be larger goals around where you intend to take your API.
Milestones give developers insight into what the plans are the future of an API, using just a title, description and target date. Milestones are one of the simplest aspects of an API, but can go a long way in creating goodwill with your developers.
API providers have to balance transparency versus giving away information to competitors, something that will take regular consideration. A transparent roadmap for your API will let developers prepare properly, while also letting them know what features are coming in the future--eliminating the possibility of competing with your own ecosystem.
Consider what you can include in your Github roadmap, and using Github for issue management.
It will be 3 months since the White House CiO Steven VanRoekel released a federal API strategy, entitled "Digital Government: Building a 21st Century Platform to Better Serve the American People", part of the Executive Order 13571, directing all federal departments and agencies to make open data, content and web APIs the new default.
This Thursday, August 23rd 2012 will be the first major milestone for departments and agencies, where they should have met two goals:
- 2.1 - Engage with customers to identify at least two existing major customer-facing services that contain high-value data or content as first-move candidates to make compliant with new open data, content, and web API policy
- 7.1 - Engage with customers to identify at least two existing priority customer-facing services to optimize for mobile use
I’ve been monitoring 246 departments and agencies and so far three have released drafts of their strategy:
|Department of Commerce|
|Department of Education (ED)|
|United States Agency for International Development (USAID)|
While the Department of Commerce and Department of Education have only published paragraphs discussing how they are engaging with users, the United States Agency for International Development (USAID) has identified three datasets for the web API portion and two optimized for mobile use. All three have published their digital strategy in HTML, XML and JSON formats, meeting the requirements of "machine readable by default".
I anticipate we’ll see many more agencies releasing their digital strategies this week, but there is no way to tell how many agencies will be able to get on board with the CIO’s strategy in time. No matter what, the next 3 months will be critical time for APIs in Washington DC, not just because of the presidential election, but the end of November will be the next milestone where agencies are expected to have established an agency-wide governance structure for developing and delivering digital services via web APIs.
If you think there is a link I should have listed here feel free to tweet it at me, or submit as a Github issue. Even though I do this full time, I'm still a one person show, and I miss quite a bit, and depend on my network to help me know what is going on.