Monday, March 22, 2010

Making APIs Faster: Introducing Partial Response and Partial Update

At Google, we strive to make the web faster. Today, we’re proud to take our first big step in making APIs faster by introducing two experimental features in the Google Data Protocol, partial response and partial update. Together, partial response and partial update can drastically reduce the network, memory, and CPU resources needed to work with Google APIs.

It’s easy to understand the benefit of partial response and partial update. Imagine that you are writing a new Android calendar widget, and you want to display the time and title of the recently changed events on your Google Calendar. With the old Calendar Data API, you would request your calendar’s events feed and receive a large amount of information in response -- including lots of extra data like the attendee list and the event description.

With the addition of partial response, however, you can now use the fields query parameter to request only relevant information -- in this case, event titles and times. Constructing such a request using the fields query parameter is simple:

GET http://www.google.com/calendar/feeds/zachpm@google.com/private/full?fields=entry(title,gd:when)

By including the entry argument and specifying title and gd:when, this request ensures that the partial response contains only the title and time for each event, along with a small amount of wrapping metadata.

But say you want to also enable the widget to change the time of calendar events. With partial update, you can easily accomplish this: simply edit the data you received in the partial response and use the HTTP PATCH verb to send the modified data back to the server. The server then intelligently interprets your PATCH, updating only the fields you chose to send. Throughout this entire read-modify-write cycle, the unneeded data remains server-side and untouched.

Now for a quick demo. If you’re currently logged into a Google account, compare the size of your full calendar feed and your partial calendar feed. When we ran this test, our full calendar feed contained 160 kB of data while the partial feed only contained 8 kB -- the partial response successfully reduced total data transfer by 95%! Performance enhancements like this are especially apparent on a mobile device, where every byte of memory and every CPU cycle count. In nearly all clients, partial response and partial update make it more efficient to send, store, parse, cache, and modify only the data that you need.

As of today, partial response and partial update are supported in four Google APIs:
... and we’re planning on adding support for most of the APIs that are built on the Google Data Protocol soon. Stay tuned for more information, and if you can’t wait, feel free to lobby for partial update and partial response in your favorite API’s public support group. And for those of you who’ll be at Google I/O this year, be sure to check out the Google API sessions that are in store.

Thanks for joining us in our effort to make APIs on the web as fast and as efficient as possible!

12 comments:

  1. LinkedIn API also does this!

    Check it out

    http://developer.linkedin.com/docs/DOC-1014

    ReplyDelete
  2. Really cool! I wish the flickr api did this...

    ReplyDelete
  3. Looking forward to seeing this on Google Spreadsheets.

    ReplyDelete
  4. It's a good news for developer who uses Google Data API to do something.

    ReplyDelete
  5. I'm using the api to access Google Docs to retrieve just the title and it's throwing an error:

    "Fields query parameter is not supported"

    for this url:

    http://docs.google.com/feeds/documents/private/full?fields=title%2Cid

    Can we make these partial requests on Google Docs? What are the supported API's so far?

    Thanks so much

    ReplyDelete
  6. ... (after reading the post): Or rather, when do you think you'll have google docs support?

    ReplyDelete
  7. Adding this to Domain Shared Contacts would make my sync script run sooooo much faster :)

    ReplyDelete
  8. really nice feature API.this is great for developers and mobile handset performance. :-)

    ReplyDelete
  9. Any idea when the Contacts API is going to support this?

    ReplyDelete
  10. A presentation on the LinkedIn API (which looks a little more complete to me) can be found at:
    http://www.slideshare.net/linkedin/building-consistent-restful-apis-in-a-highperformance-environment

    I have done something similar with some JAX-RS integration with Ebean ORM (Java) (inspired by the LinkedIn API).

    http://www.avaje.org/ebean/jaxrs.html

    This is JSON only at the moment.

    ReplyDelete
  11. This has a fancy name, but the concept is common in SQL, using "select field1, field2,..." instead of "select * ... "

    ReplyDelete
  12. i've found the one from linkedIn pretty dev-friendly

    ReplyDelete