Get started at no cost with a faster, larger Cloud SQL database

By Joe Faith, Product Manager

Cross-posted with the Official Google Enterprise Blog

You want your applications to be fast, even with millions of users. Anytime your user tries to retrieve information from the app or update settings, it should happen instantly. For the best performance, you need faster, larger databases - especially if you have a growing user base to serve.

Google App Engine is designed to scale. And now Google Cloud SQL—a MySQL database that lives in Google’s cloud—has new features to meet the demand for faster access to more data. With today’s updates, you can now work with bigger, faster MySQL databases in the cloud:

• More Storage: We’re increasing the available storage on Cloud SQL to 100GB – ten times more than what used to be available.
  
• Faster Reads: We’re increasing the maximum size of instances to 16GB RAM, a 4 times increase in the amount of data you can cache.
  
• Faster Writes: We’re adding functionality for optional asynchronous replication, which gives the write performance of a non-replicated database, but the availability of a replicated one.
  
• EU datacenter availability: Now you can choose to store your data and run your Cloud SQL database instance in either our US or EU data centers.
  
• Integration with Google Apps Script: We’re making it quick and easy for businesses using Google Apps to use Cloud SQL. Publish and share data with Google Sheets, add data to Google Sites pages or create simple Google Forms without worrying about hosting or configuring servers. 

Introducing a new trial offer 
Many of you have requested a trial offer to test out Cloud SQL. Today, we’re introducing a 6- month trial offer at no charge, effective until June 1, 2013. This will include one Cloud SQL instance with 0.5 GB of storage. Sign up now and get started on Cloud SQL at no cost.


Joe Faith is a Product Manager on the Google Cloud Team. In a previous life he was a researcher in machine learning, bioinformatics, and information visualization, and was founder of charity fundraising site Fundraising Skills.

Posted by Scott Knaster, Editor

Streak brings CRM to the inbox with Google Cloud Platform

By Aleem Mawani, Co-Founder of Streak

Cross-posted with the Google App Engine Blog

This guest post was written by Aleem Mawani, Co-Founder of Streak, a startup alum of Y Combinator, a Silicon Valley incubator. Streak is a CRM tool built into Gmail. In this post, Aleem shares his experience building and scaling their product using Google Cloud Platform.

Everyone relies on email to get work done – yet most people use separate applications from their email to help them with various business processes. Streak fixes this problem by letting you do sales, hiring, fundraising, bug tracking, product development, deal flow, project management and almost any other business process right inside Gmail. In this post, I want to illustrate how we have used Google Cloud Platform to build Streak quickly, scalably and with the ability to deeply analyze our data.

We use several Google technologies on the backend of Streak:

• App Engine to serve our app.

• App Engine Datastore to persist user data.

• Memcache to make operations fast.

• BigQuery to analyze our logs and power dashboards.

• App Engine Search API to let users sift through their data.

• Prediction API to machine learn over user data.

• Google Translate API to translate our app to over 40 languages.

Our core learning is that you should use the best tool for the job. No one technology will be able to solve all your data storage and access needs. Instead, for each type of functionality, you should use a different service. In our case, we aggressively mirror our data in all the services mentioned above. For example, although the source of truth for our user data is in the App Engine Datastore, we mirror that data in the App Engine Search API so that we can provide full text search, Gmail style, to our users. We also mirror that same data in BigQuery so that we can power internal dashboards.

System Architecture

App Engine - We use App Engine for Java primarily to serve our application to the browser and mobile clients in addition to serving our API. App Engine is the source of truth for all our data, so we aggressively cache using Memcache. We also use Objectify to simplify access to the Datastore, which I highly recommend.

Google Cloud Storage - We mirror all of our Datastore data as well as all our log data in Cloud Storage, which acts as a conduit to other Google cloud services. It lets us archive the data as well as push it to BigQuery and the Prediction API.

BigQuery - Pushing the data into BigQuery allows us to run non-realtime queries that can help generate useful business metrics and slice user data to better understand how our product is getting used. Not only can we run complex queries over our Datastore data but also over all of our log data. This is incredibly powerful for analyzing the request patterns to App Engine. We can answer questions like:

• Which requests cost us the most money?
• What is the average response time for every URL on our site over the last 3 days?

BigQuery helps us monitor error rates in our application. We process all of our log data with debug statements, as well as something called an “error type” for any request that fails. If it’s a known error, we'll log something sensible, and we log the exception type if we haven’t seen it before. This is beneficial because we built a dashboard that queries BigQuery for the most recent errors in the last hour grouped by error type. Whenever we do a release, we can monitor error rates in the application really easily.

A Streak dashboard powered by BigQuery showing current usage statistics

In order to move the data into Cloud Storage from the Datastore and LogService, we developed an open source library called Mache. It’s a drop-in library that can be configured to automatically push data into BigQuery via Cloud Storage. The data can come from the Datastore or from LogService and is very configurable - feel free to contribute and give us feedback on it!

Google Cloud Platform also makes our application better for our users. We take advantage of the App Engine Search API and again mirror our data there. Users can then query their Streak data using the familiar Gmail full text search syntax, for example, “before:yesterday name:Foo”. Since we also push our data to the Prediction API, we can help users throughout our app by making smart suggestions. In Streak, we train models based on which emails users have categorized into different projects. Then, when users get a new email, we can suggest the most likely box that the email belongs to.

One issue that arises is how to keep all these mirrored data sets in sync. It works differently for each service based on the architecture of the service. Here’s a simple breakdown:

Having these technologies easily available to us has been a huge help for Streak. It makes our products better and helps us understand our users. Streak’s user base grew 30% every week for 4 consecutive months after launch, and we couldn’t have scaled this easily without Google Cloud Platform. To read more details on why Cloud Platform makes sense for our business, check out our case study and our post on the Google Enterprise blog.


Aleem Mawani is the co-founder of Streak.com, a CRM tool built into Gmail. Previously, Aleem worked on Google Drive and various ads products at Google. He has a degree from the University of Waterloo in Software engineering and an MBA from Harvard University.

Posted by Scott Knaster, Editor

Now in BigQuery: batch queries and a connector for Excel

By Ryan Boyd, Developer Advocate for Cloud Data Services

Businesses and developers are using BigQuery to solve a wide variety of use cases – from optimizing advertising campaigns, to spotting inventory shortfalls, to understanding customer behavior. Accommodating these varied use cases requires BigQuery to be flexible, both for the developers integrating applications with the API and for the analysts running ad-hoc queries. Today we’ve made it more flexible by adding batch queries and a connector for Microsoft Excel.

Batch priority queries

BigQuery was designed for ad-hoc, iterative analytics on millions-to-billions of rows of data. When you’re diving into your data to gain insights, you want your queries to run in seconds rather than waiting minutes or hours. Sometimes our customers don’t need these fast responses when they’re running nightly jobs to update dashboards or reports, but want to use the same BigQuery technology and underlying datasets for these queries. We’ve now added batch pricing to accommodate these developers, allowing them to run their queries at a significantly lower cost.

Here’s how to set the priority to ‘batch’ when submitting a new query via the Google APIs Client Library for Java:

    Job job = new Job();
    JobConfiguration config = new JobConfiguration();
    JobConfigurationQuery queryConfig = new JobConfigurationQuery();
    config.setQuery(queryConfig);

    job.setConfiguration(config);
    queryConfig.setQuery(querySql);
    queryConfig.setPriority("BATCH");

    com.google.api.services.bigquery.Bigquery.Jobs.Insert insert =
     bigquery.jobs().insert(projectId, job);

Batch queries will execute between 30 minutes and 3 hours after they are submitted. See more information in our Developers Guide.

BigQuery Connector for Excel

Spreadsheets are a popular tool for analysts, executives and and developers to explore data. Last year we launched the ability for users of Google Spreadsheets to execute BigQuery queries using the Google Apps Script integration. Today, we’re launching the BigQuery Connector for Excel, which allows Microsoft Excel users to do the same with the ‘External Data’ functionality built into the product. Once the BigQuery results are in Excel, you can easily make pivot tables, create charts and integrate it with data from other sources. If you’re interested, you can try it right now!

Let us know what you think of these new features and what else you’d like to see in the roadmap by reaching out on Google+. We’ll also be holding office hours this Friday at 10 AM PDT on Google Developers Live to talk about these new features and answer any questions you have about BigQuery.

Microsoft and Excel are registered trademarks of Microsoft Corporation


Ryan Boyd is a Developer Advocate, focused on cloud data services. He's been at Google for 6 years and previously helped build out the Google Apps ISV ecosystem. He published his first book "Getting Started with OAuth 2.0" with O'Reilly.

Posted by Scott Knaster, Editor

Helping developers build more applications on Google BigQuery

Amanda

Ju-kay

By Ju-kay Kwek and Amanda Bradford, Google BigQuery Team

In May we launched Google BigQuery, a fully managed cloud-based service that enables businesses to analyze enormous amounts of data in the cloud. While we're continually amazed by the range of business problems being solved, we recognize that writing one-off scripts to ingest data, or creating custom front-end integration, requires effort and takes away time from the fun stuff: getting results.

So today we're pleased to highlight a few new members of our Cloud Platform Partner Program that are here to help you be even more productive – by delivering tools integrated with BigQuery that make it much easier to automatically load data from a broad set of sources, as well as to analyze and visualize the data with spectacular dashboards.

Import data from multiple sources into BigQuery

We’ve partnered with Informatica, Pervasive Software, Talend and SQLstream to make it easier to bring data from a variety of sources into BigQuery. This means you can use their BigQuery connectors to move data very easily from on-premise or cloud IT systems to BigQuery. For example, TribusPoint, a consulting firm, leveraged the Informatica Cloud BigQuery connector to rapidly move large files from their on-premise data centers to BigQuery.

Build rich interactive dashboards on BigQuery

We’ve partnered with data visualization providers QlikTech, Jaspersoft, Bime Analytics and Metric Insights to help you build rich, interactive dashboards for a broad range of customers. You can use their tools to build dashboards and reports very easily. For example, Pixelfish leveraged Metric Insights BigQuery integration to create dashboards that delivered a 300% improvement in customer engagement.

Click the partner links above to see more specific customer examples of each. We have just scratched the surface on Google BigQuery. We can’t wait to see what other cool applications you can build on Google BigQuery using our APIs. Hack away!


Ju-kay Kwek is the Product Management Lead for Google's Cloud Big Data initiative. In this role, he focuses on creating services that enable businesses and developers to harness Google's unparalleled data processing infrastructure and algorithms to tackle Big Data needs.

Amanda Bradford works in Business Development at Google, driving strategic alliances and partnerships for Google and specializing in the Google Cloud Platform.

Posted by Scott Knaster, Editor

Google Compute Engine: Computing without limits

By Craig McLuckie, Product Manager, Google Compute Engine

Over the years, Google has built some of the most high performing, scalable and efficient data centers in the world by constantly refining our hardware and software. Since 2008, we've been working to open up our infrastructure to outside developers and businesses so they can take advantage of our cloud as they build applications and websites and store and analyze data. So far this includes products like Google App Engine, Google Cloud Storage, and Google BigQuery.

Today, in response to many requests from developers and businesses, we're going a step further. We're introducing Google Compute Engine, an Infrastructure-as-a-Service product that lets you run Linux Virtual Machines (VMs) on the same infrastructure that powers Google. This goes beyond just giving you greater flexibility and control; access to computing resources at this scale can fundamentally change the way you think about tackling a problem.

Google Compute Engine offers:

• Scale. At Google we tackle huge computing tasks all the time, like indexing the web, or handling billions of search queries a day. Using Google's data centers, Google Compute Engine reduces the time to scale up for tasks that require large amounts of computing power. You can launch enormous compute clusters - tens of thousands of cores or more.
• Performance. Many of you have learned to live with erratic performance in the cloud. We have built our systems to offer strong and consistent performance even at massive scale. For example, we have sophisticated network connections that ensure consistency. Even in a shared cloud you don’t see interruptions; you can tune your app and rely on it not degrading.
• Value. Computing in the cloud is getting even more appealing from a cost perspective. The economy of scale and efficiency of our data centers allows Google Compute Engine to give you 50% more compute for your money than with other leading cloud providers. You can see pricing details here.

The capabilities of Google Compute Engine include:

• Compute. Launch Linux VMs on-demand. 1, 2, 4 and 8 virtual core VMs are available with 3.75GB RAM per virtual core.
• Storage. Store data on local disk, on our new persistent block device, or on our Internet-scale object store, Google Cloud Storage.
• Network. Connect your VMs together using our high-performance network technology to form powerful compute clusters and manage connectivity to the Internet with configurable firewalls.
• Tooling. Configure and control your VMs via a scriptable command line tool or web UI. Or you can create your own dynamic management system using our API.

At launch, we have worked with a number of partners - such as RightScale, Puppet Labs, OpsCode, Numerate, Cliqr and MapR - to integrate their products with Google Compute Engine. These partners offer management services that make it easy for you to move your applications to the cloud and between different cloud environments.

You can learn more about Google Compute Engine here. We’re going to pace ourselves and start with Google Compute Engine in limited preview (sign up here), but our goal is to give you all the pieces you need to build anything you want in the cloud. Whether you need a platform like Google App Engine, or virtual machines like Google Compute Engine, these days, you define your limits. We’re just at the start of what the cloud can do.


Craig McLuckie is the Product Management Lead for Google Compute Engine. He spends his days working with an amazing engineering team to open Google’s infrastructure to the world.

Posted by Scott Knaster, Editor

Pricing plan announced for Google Cloud SQL

By Joe Faith, Product Manager

Google Cloud SQL provides a fully managed database service for Google App Engine applications. Hosted on Google's infrastructure and based on the familiar MySQL database, Google Cloud SQL automatically provisions and maintains your databases, allowing you to focus on your applications and services.

In March, we were delighted to welcome our 10,000th developer on Google Cloud SQL, joining businesses like Daffodil, who halved their development time by building on Google's platform.

Since the preview launch in October 2011, we’ve been busy working on improving the performance, adding features like scheduled backups, and multihoming to increase availability and improve performance. We are also now offering more powerful instances with up to 4GB of RAM. Today, we are announcing our pricing, with two options to choose from:

• For developers who want to try out the service, or who have lightweight applications, we offer a flexible "per use" pricing scheme. For example, you can get started with a cloud hosted MySQL database for around a dollar per month. You pay for just what you use.
• For developers with more traffic, there are package plans that are more economical and help you predict your costs in advance.

We will not start charging for the service until June 12th. Full details of the pricing plans are available here: https://developers.google.com/cloud-sql/docs/billing

Google Cloud SQL is currently in limited preview. If you want to give us a try, start here: https://developers.google.com/cloud-sql/.


Joe Faith is a Product Manager on the Google Cloud Team. In a previous life he was a researcher in machine learning, bioinformatics, and information visualization, and was founder of charity fundraising site Fundraising Skills.

Posted by Scott Knaster, Editor

Using Google BigQuery to learn from GitHub data

By Ilya Grigorik, Web Performance Engineer

Open-source developers all over the world contribute to millions of projects every day: writing and reviewing code, filing and discussing bug reports, updating documentation and project wikis, and so forth. The data generated from this activity can reveal interesting trends across many industries, including popularity of programming languages over time, defect rates, contribution metrics, and popularity of specific frameworks and libraries.

The challenge in extracting these trends is gathering the data. Each project has its own distributed workflow, code repositories, and conventions. Having hosted dozens of my own projects on GitHub, I've long wanted to analyze the developer activity from the 2.6M+ public projects hosted on GitHub. Hence, earlier this year GitHub Archive was born!

GitHub Archive is a project to record the public GitHub timeline, archive it, and make it easily accessible for further analysis. Each day it archives over 120,000 public activities, ranging from new commits and fork events to opening and closing tickets, each with detailed metadata.

Once I collected the data, I needed a tool to analyze it, and that is when I found Google BigQuery. Based on the research behind Dremel, a popular internal tool at Google for analyzing web-scale datasets, BigQuery allowed me to easily import the entire dataset and use a familiar SQL like syntax to comb through the gigabytes of data in seconds. Plus the tool will scale to terabyte datasets, so there is plenty of room to grow!

The best news is that thanks to collaboration from the GitHub and BigQuery teams, the GitHub dataset is now public and available for you to slice and dice in any way you like. No need to worry about data gathering or database schemas: BigQuery will do all the heavy lifting, and you can just compose your queries to be executed in realtime.

Here's a real-world example. What are the most popular programming languages on GitHub over the past month?

If you are curious for more, sign up for BigQuery and follow the instructions on githubarchive.org to access the GitHub dataset. You can use the free 100GB query quota to run your analysis and perhaps even win some of the prizes from the GitHub Data Challenge!


Ilya Grigorik is a Web Performance Engineer and Advocate at Google, an open-source evangelist, and an analytics geek. You can find him on GitHub under igrigorik, and blogging about web performance at igvita.com.

Posted by Scott Knaster, Editor

Google BigQuery brings Big Data analytics to all businesses

By Ju-kay Kwek, Product Manager, BigQuery

BigQuery enables businesses and developers to gain real-time business insights from massive amounts of data without any upfront hardware or software investments. Imagine a big pharmaceutical company optimizing daily marketing spend using worldwide sales and advertisement data. Or think of a small online retailer that makes product recommendations based on user clicks. Today, we are making BigQuery publicly available, an important milestone in our effort to bring Big Data analytics to all businesses via the cloud.

Since announcing BigQuery in limited preview last November, many businesses and developers have started using it for real-time Big Data analytics in the cloud. Claritics, a social and mobile analytics company, built a web application for game developers to gain real-time insights into user behavior. Crystalloids, an Amsterdam-based analytics firm, built a cloud-based application to help a resort network analyze customer reservations, optimize marketing and maximize revenue. This just scratches the surface of use cases for BigQuery.

BigQuery is accessible via a simple UI or REST interface. It lets you take advantage of Google’s massive compute power, store as much data as needed and pay only for what you use. Your data is protected with multiple layers of security, replicated across multiple data centers and can be easily exported.

Developers and businesses can sign up for BigQuery online and query up to 100 GB of data per month for free. See our introductory pricing plan for storing and querying datasets of up to 2 TB. If you need more than that, contact a sales representative.

We hope you will be able to gain real-time business insights using BigQuery. Share your BigQuery use cases and feedback in our user forums or on our +Google Enterprise page.


Ju-kay Kwek is the Product Management Lead for Google's Cloud Big Data initiative. In this role, he focuses on creating services that enable businesses and developers to harness Google's unparalleled data processing infrastructure and algorithms to tackle Big Data needs.

Posted by Scott Knaster, Editor

Fridaygram: goodbye to 2011

By Scott Knaster, Google Code Blog Editor

This is the last Fridaygram of 2011, and like most everybody else, we’re in a reflective mood. It’s also the 208th post on Google Code Blog this year, which means we’ve averaged more than one post every two days, so that’s plenty of stuff for you to read. What did we write about?

At Google, we love to launch. Many of our posts were about new APIs and client libraries. We also posted a bunch of times about HTML5 and Chrome and about making the web faster. And we posted about Android, Google+, and Google Apps developer news.

Many of our 2011 posts were about the steady progress of App Engine, Cloud Storage, and other cloud topics for developers. We also published several times about commerce and in-app payments.

2011 was a stellar year for Google I/O and other developer events around the world. Some of our most popular posts provided announcements, details, and recaps of these events. And we welcomed a couple dozen guest posts during Google I/O from developers with cool stories to tell.

The two most popular Code Blog posts of the year were both launches: the Dart preview in October, and the Swiffy launch in June.

Last, and surely least, I posted 26 Fridaygrams in an attempt to amuse and enlighten you. Thank you for reading those, and thanks for dropping by and reading all the posts we’ve thrown your way this year. See you in 2012!

And finally, please enjoy one more Easter egg.

Google Prediction API: faster, easier to use, and more accurate

By Marc Cohen, Developer Relations

This holiday season, the Google Prediction API Team is bringing you four presents and, thanks to the joys of cloud computing, no reindeer are required for delivery. Here’s what you’ve already received:

• Faster on-ramp: We’ve made it easier to get started by enabling you to create an empty model (by sending a trainedmodels.insert request with no storageDataLocation specified) and add training data using the trainedmodels.update method. This change allows you to submit your model contents without needing to stage the data in Google Cloud Storage.
• Improved updates: The algorithms used to implement model updates (adding additional data to existing models) have been modified to work faster than ever.
• More classification algorithms: We’ve increased the number of classification algorithms used to build predictive models, resulting in across-the-board improvements in accuracy.
• Integration with Google Apps Script: Prediction services are now available as part of Google Apps Script, which means you can integrate prediction services with Google Docs, Google Maps, Gmail, and other great Google products.

All of the above enhancements are supported by the current Prediction API version 1.4 so you can enjoy these features using the existing client libraries.

Happy Holidays from the Google Prediction API Team. We’re looking forward to bringing you more exciting features in 2012!


Marc Cohen is a member of Google’s Developer Relations Team in Seattle. When not teaching Python programming and listening to indie rock music, he enjoys using the Google Prediction API to peer into the future.

Posted by Scott Knaster, Editor

Introducing Au-to-do, a sample application built on Google APIs

By Dan Holevoet, Developer Relations Team

A platform is more than the sum of its component parts. You can read about it or hear about it, but to really learn what makes up a platform you have to try it out for yourself, play with the parts, and discover what you can build.

With that in mind, we started a project called Au-to-do: a full sample application implementing a ticket tracker, built using Google APIs, that developers can download and dissect.

Au-to-do currently uses the following APIs and technologies:

• Google App Engine (with the Python runtime, Datastore API, and Task Queues API)
• Google Cloud Storage
• Google Prediction API
• Google Tasks API
• OAuth 2.0

Additional integrations with Google APIs are on their way. We are also planning a series of follow-up blog posts discussing each of the integrations in depth, with details on our design decisions and best practices you can use in your own projects.

By the way, if you’re wondering how to pronounce Au-to-do, you can say "auto-do" or "ought-to-do" — either is correct.

Ready to take a look at the code? Check out the getting started guide. Found a bug? Have a great idea for a feature or API integration? Let us know by filing a request.

Happy hacking!


Dan Holevoet joined the Google Developer Relations team in 2007. When not playing Starcraft, he works on Google Apps, with a focus on the Calendar and Contacts APIs. He's previously worked on iGoogle, OpenSocial, Gmail contextual gadgets, and the Google Apps Marketplace.

Posted by Scott Knaster, Editor

Simplifying Access Control in Google Cloud Storage

By Navneet Joneja, Product Manager

Google Cloud Storage is a robust, high-performance service that enables developers and businesses to use Google’s infrastructure to power their data. Today, we’re announcing a new feature that makes it even easier to control and share your data.

Per-Bucket Default Object ACLs

Customers building a wide variety of applications have asked us for an easier mechanism to control the permissions granted on newly created objects. Now you can define your access control policy for a bucket once by specifying a Default Object ACL for any bucket, and we’ll automatically apply that ACL to any object without an explicitly defined ACL. You can always override the default by providing a canned ACL when you upload the object or by updating the object’s ACL afterwards. This mechanism simplifies wide variety of use cases, including data sharing, controlled-access data sets and corporate drop-boxes.

New buckets without Default ACLs

After analyzing how customers use our service, we’ve also decided to make a few small changes to the behavior of buckets that have no explicit default object ACL. Effective today, new buckets are created with an implied project-private default object ACL. In other words, project editors and owners will have FULL_CONTROL access to new objects, and project viewers will have READ access to them. This change better aligns the default behavior with how our customers use storage. You can change a bucket’s default object ACL at any time after creating the bucket.

Existing buckets have an effective default object ACL of "private", and they will continue to work as they always have until and unless you specify a new default object ACL for them.


Navneet Joneja loves being at the forefront of the next generation of simple and reliable software infrastructure, the foundation on which next-generation technology is being built. When not working, he can usually be found dreaming up new ways to entertain his intensely curious one-year-old.

Posted by Scott Knaster, Editor

App Engine 1.6.0 out of Preview release

By By Greg D'Alesandre, App Engine team

Three and a half years after App Engine's first Campfire One, App Engine has graduated from Preview and is now a fully supported Google product. We started out with the simple philosophy that App Engine should be "easy to use, easy to scale, and free to get started". And with 100 billion+ monthly hits, 300,000+ active apps, and 100,000+ developers using our product every month, it's clear that this philosophy resonates. Thanks to your support, Google is making a long term investment in App Engine.

When we announced our plans to leave Preview earlier this year, we made a commitment to improving the service by adding support for Python 2.7, Premier Accounts, and Backends as well as several changes launching today:

• Pricing: The new pricing structure announced in May (and updated based on feedback from the community) will now be reflected in your bill starting Nov 7.
• Terms of Service: We have a new terms of service, including a 3 year deprecation policy, effective today.
• Service Level Agreement: All paid applications on the High Replication Datastore are covered by our SLA.

We are holding a series of App Engine Office Hours via Google+ this week for anyone with questions about how this new pricing impacts your application. The list of times can be found on the Google Developers events page, with links to join the hangout while the office hours are scheduled. Also, please don't hesitate to contact us at appengine_updated_pricing@google.com with any questions or concerns.

You can read the full details of our release on the App Engine blog. We'd like to thank you for investing in our platform for the last three years. We look forward to what the future will bring.


Greg D'Alesandre is now the Senior Product Manager for App Engine after coming back from riding the Google Wave in Sydney. And he's obsessed with chocolate, no, seriously, obsessed.

Posted by Scott Knaster, Editor

App Engine Premier Accounts and a new release

By Greg D'Alesandre, App Engine team

Cross-posted from the Google App Engine Blog

2011 has seen some exciting releases for App Engine. As the days get shorter, the weather gets colder, and all that Halloween candy starts tempting everyone in the grocery store, we’ve been hard at work on our latest action-packed release.

Premier Accounts

When choosing a platform for your most critical business applications, we recognize that uptime guarantees, easy management and paid support are often just as important as product features. So today we’re launching Google App Engine premier accounts.  For $500 per month (not including the cost to provision internet services), you’ll receive:

• Premium support (see the 
Technical Support Services Guidelines for details).
• A 99.95% uptime Service Level Agreement (see the draft agreement; the final agreement will be in the signed offline agreement).
• The ability to create an unlimited number of apps on your premier account domain.
• No minimum monthly fees per app. Pay only for the resources you use.
• Monthly billing via invoice.

To sign up for a premier account, please contact our sales team at appengine_premier_requests@google.com.  

Python 2.7

PIL? NumPy? Concurrent requests? Python 2.7 has it all, and today we’re opening up Python 2.7 as an experimental release. We’ve put together a list of all the known differences between the current 2.5 runtime and the new runtime.

Overall Changes

We know that bumping up against hard limits can be frustrating, and we’ve talked all year about our continued push to lift our system limits. With this release we are raising several of these:

• Request Duration: The frontend request deadline has been increased from 30 seconds to 60 seconds. We’ve increased the maximum URLFetch deadline to match from 10 seconds to 60 seconds.
• File limits: We’ve increased the number of files you can upload with your application from 3,000 to 10,000 files, and the file size limit has also been increased from 10MB to 32MB.
• API Limits: Post payloads for URLFetches are now capped at 5MB instead of 1MB.

We’re also announcing several limited preview features and trusted tester programs:

• Cloud SQL Preview: We announced last week that we are offering a preview of SQL support in App Engine. Give it a try and let us know what you think.
• Full-text Search: We are looking for early trusted testers for our long-anticipated Full-Text Search API. Please fill out this form if you’re interested in trying it out.
• Conversion API: Ever wanted to convert from text to PDF in your App? Then consider signing up as a trusted tester for the Conversion API.

Datastore

• Cross Group (XG) Transactions: For those who need transactional writes to entities in multiple entity groups (and that's everyone, right?), XG Transactions are just the thing. This feature uses two phase commit to make cross group writes atomic just like single group writes.

Platform Improvements

• Experimental Google Cloud Storage Integration: As Google Cloud Storage, formerly Google Storage for Developers, graduates from labs, we are improving our integration by adding access via the files API.
• Prediction API: Another of our friends graduating from labs is the Prediction API. Check out some of the App Engine examples in their getting started guide.

Of course, these are just the high level changes. This release is packed full of features and bug fixes, and as always, we welcome your feedback in the group.

Greg D'Alesandre is now the Senior Product Manager for App Engine after coming back from riding the Google Wave in Sydney. And he's obsessed with chocolate, no, seriously, obsessed.

Posted by Scott Knaster, Editor

Google Cloud Storage is out of Code Labs, with new features and lower price

By Navneet Joneja, Product Manager for Google Cloud Storage

Google Storage for Developers is now out of Code Labs, and has a new name: Google Cloud Storage. In addition, we're also happy to announce some new features, and a significant price reduction.

App Engine File API Support

When we opened the service to all this summer, many of our customers asked for an easier way to use Google Cloud Storage with their App Engine applications. In response to your feedback, you can now read and write your data via the App Engine Files API, enabling you to quickly build your content management tools, data sharing applications, web games and more using the powerful combination of App Engine and Cloud Storage. This feature is experimental and currently Python-only, but we’re working on adding Java support and additional features.

Usage Information

We’re introducing a new API that gives you access to detailed usage information (including network access and storage use data). You can use this feature to analyze your usage, integrate with your analysis systems and build your own value-added applications using Google Cloud Storage. This feature is currently experimental.

Lower Prices

We're no longer charging for upload bandwidth into the Google cloud. In addition, we’re lowering our prices across the board and introducing volume discounts for our larger users. We are committed to offering an extremely high quality of service to all our customers. As the product has evolved, we’ve found ways to offer the same great service at a lower cost, so now our prices are lower too. For example, under our new prices, a customer storing a hundred terabytes of data, reading twenty terabytes and writing ten terabytes a month would pay approximately 40% less a month. The difference is even greater for customers with higher usage. Our new prices are retroactive to the beginning of October. Please see our updated pricing here.

As always, we welcome your feedback in our discussion group. If you haven’t yet tried Google Cloud Storage, you can sign up and get started here.


Navneet Joneja loves being at the forefront of the next generation of simple and reliable software infrastructure, the foundation on which next-generation technology is being built. When not working, he can usually be found dreaming up new ways to entertain his intensely curious one-year-old.

Posted by Scott Knaster, Editor

Google Cloud SQL: your database in the cloud

By Navneet Joneja, Product Manager for Google Cloud SQL

Cross-posted from the Google App Engine Blog

One of App Engine’s most requested features has been a simple way to develop traditional database-driven applications. In response to your feedback, we’re happy to announce the limited preview of Google Cloud SQL.

You can now choose to power your App Engine applications with a familiar relational database in a fully-managed cloud environment. This allows you to focus on developing your applications and services, free from the chores of managing, maintaining and administering relational databases.

Google Cloud SQL brings many benefits to the App Engine community:

• No maintenance or administration - we manage the database for you.
• High reliability and availability - your data is replicated synchronously to multiple data centers. Machine, rack and data center failures are handled automatically to minimize end-user impact.
• Familiar MySQL database environment with JDBC support (for Java-based App Engine applications) and DB-API support (for Python-based App Engine applications).
• Comprehensive user interface for administering databases.
• Simple and powerful integration with Google App Engine.

The service includes database import and export functionality, so you can move your existing MySQL databases to the cloud and use them with App Engine.

Cloud SQL is available free of charge for now, and we will publish pricing at least 30 days before charging for it. The service will continue to evolve as we work out the kinks during the preview, but let us know if you’d like to take it for a spin.


Navneet Joneja loves being at the forefront of the next generation of simple and reliable software infrastructure, the foundation on which next-generation technology is being built. When not working, he can usually be found dreaming up new ways to entertain his intensely curious one-year-old.

Posted by Scott Knaster, Editor

Google @ GDC Online Oct. 10th-12th

By Amy Walgenbach, Developer Marketing

This year at the Game Developers Conference (GDC) Online we have organized a Developer Day on Oct. 10th full of Google information for game developers. It will feature hardcore technical information on Google products and platforms delivered by Google engineers and developer advocates. We’ll discuss the latest projects we’re working on and how our online technologies can help you better create, distribute, and monetize games that reach a larger audience than ever before. We’ll present everything from how developers can build hardware accelerated 3D games for the browser with WebGL to the game framework used to bring Angry Birds to the Web.

In addition to the Developer Day, we will also have a booth on the Expo floor on Oct. 11th-12th where we’ll have representatives from the Chrome Web Store, Native Client, WebGL, App Engine, Google+, In-App Payments, Google TV, and AdSense/AdMob demoing technologies and platforms for game developers. Come by booth 503 to try out Google products and ask questions, or hang out in our Google TV lounge.

For more information on our presence at GDC Online, including session and speaker details, please visit http://www.google.com/events/gdc/2011. Hope to see you in Austin!

Not able to attend GDC? Check out Google Game Developer Central to get an overview of Google products and services that are particularly relevant to game developers.

Amy Walgenbach is the Product Marketing lead for the Google+ platform and leads developer marketing for games at Google.

Posted by Scott Knaster, Editor

Android Meet App Engine, App Engine Meet Android

By Chris Ramsdale, Product Manager, GWT and GPE

Imagine this: you've spent the past few months hammering away at the latest mobile game sensation, Mystified Birds, and you are one level away from complete mastery. And then it happens. In a fit of excitement you throw your hands up, and along with them your Nexus S, which settles nicely at the bottom of the pool you happen to be relaxing next to. The phone is rendered useless. Luckily, your insurance policy covers the replacing the device and the Android Market handles replacing your apps. Unluckily though, all of your Mystified Birds data went the way of your device, leaving you to start from scratch.

Wouldn't it be great if your new device not only contained all of your apps, but all of your valuable data as well? We think so. With Google Plugin for Eclipse (GPE) v2.4 it's much easier to build native Android apps that can take data with them wherever they go. And there's no better place to host your backend service and store your data than Google's cloud service, App Engine.

With the latest release of GPE, we're bringing together these two great Google platforms, Android and App Engine, with a set of easy-to-use developer tools. Diving a bit deeper, here are some of the features offered in GPE 2.4:

Project Creation

With GPE 2.4, you now have the ability to create App Engine-connected Android projects. This new Eclipse project wizard generates fully functioning Android and GWT clients that are capable of talking to the same App Engine backend using the same RPC code and business logic.

Cloud to Device Messaging Support

Polling for backend changes on a mobile device is inefficient and will result in poor app performance and battery drain. As a solution for Android developers, the Android team built Cloud to Device Messaging (C2DM), a service for sending lightweight pings to notify apps when they have pending data. We heard back from developers that integrating with C2DM results in a lot of boilerplate (and sometimes fragile) code that they would rather not maintain. With the 2.4 release of GPE, when you create a new App Engine connected Android project, you'll get this code for free. All you have to do is hook up the app-specific code to customize the handling of the C2DM notification.

RPC Generation and Tooling

Writing and maintaining RPC code (code that allows your app to communicate with backend servers) is monotonous and error prone. Let's face it, you're a mobile developer and the last thing you want to be spending time on is writing (or debugging) this type of code. In GPE 2.4 we're introducing tooling that removes this task for you, and will generate all of the underlying RPC boilerplate code within a few clicks. You specify the model objects that will be used between client and server, and GPE generates the RPC service, DTOs, and client-side calling code. To make this even better, the generated code works across Android and GWT apps, so any future changes that you make will only need to be made once.

Want to get started? Download GPE 2.4 Beta here. Note that you'll need to install the Android Developer Tools (ADT) plugin as a prerequisite, which can be found here.

If you have any feedback, we'd love to hear it and the GPE Group is the right place to submit it. The App Engine and Android Developer Groups are also great sources of information.


Chris Ramsdale is Product Manager for GWT and GPE: cramsdale@google.com

Posted by Scott Knaster, Editor

The Go programming language is coming to Google App Engine

By David Symonds, Nigel Tao, and Andrew Gerrand of the Go Team

The growing ranks of Go programmers will soon have another platform: Go will be the third language, after Python and Java, in which to write Google App Engine apps. This is an exciting new experimental feature of App Engine and a major milestone for Go.

Go is an open source language, initially designed at Google, that was released in November 2009 and has seen significant development since launch. It is a statically typed, compiled language with a dynamic and lightweight feel. It’s also an interesting new option for App Engine because Go apps will be compiled to native code, making Go a good choice for more CPU-intensive tasks. Plus the garbage collection and concurrency features of the language, combined with excellent libraries, make it a great fit for web apps.

As of today, the App Engine SDK for Go is available for download, and we will soon enable deployment of Go apps into the App Engine infrastructure. If you’re interested in starting early, sign up to be first through the door when we open it up to early testers. Once it proves solid, we’ll open it up to everyone, although it will remain an experimental App Engine feature for a while.

You don’t need an existing Go installation; the SDK is fully self-contained, so it’s very simple to get a local web app up and running. The SDK is a really easy way to start playing with Go.

More details can be found at the Go Programming Language blog.

Posted by Scott Knaster, Editor

Google App Engine: a new version and a preview of future plans

By Greg D’Alesandre, Senior Product Manager, Google App Engine

Google App Engine, which provides the ability to develop and host applications on Google’s infrastructure, has grown tremendously since it launched in preview status in 2008. More than 100,000 developers use App Engine every month to deliver apps that dynamically scale with usage without the need to manage hardware or software. App Engine now hosts more than 200,000 active apps that serve over 1.5 billion site views daily.

Over the last three years, we’ve collected great feedback from our customers and now believe that the biggest thing we can do to help our customers is to graduate App Engine from preview status. When App Engine graduates from preview status, which we expect to do in the second half of this year, we’ll add additional enterprise-grade features that allow us to support many more business application scenarios. Graduation from preview status also indicates a longer term commitment by Google to the product and provides a deprecation policy whereby we will support prior versions of product APIs for a guaranteed amount of time, allowing applications written to prior API specifications to continue to function.

Today, we’re moving forward with our business focus with the release of App Engine 1.5.0 which includes Backends, improved Task Queues, a new experimental Go runtime, and more:

• With Backends (for both Python and Java), App Engine can now support applications that require long running and high memory processes. This feature allows for new classes of applications such as report generation apps and custom search engines to be hosted on the platform.
• The improvements to Task Queues allow for applications to control how tasks are executed and easily share the work using the new REST-based APIs. This API access expands App Engine’s compatibility with other on-premise and cloud services, furthering our commitment to an open development platform.
• With 1.5.0 we are launching an experimental runtime for the Go Programming Language. Go is an open source, statically typed, compiled language with a dynamic and lightweight feel. It’s also an interesting new option for App Engine because Go apps will be compiled to native code, making Go a good choice for more CPU-intensive tasks. As of today, the App Engine SDK for Go is available for download, and we will soon enable deployment of Go apps into the App Engine infrastructure. If you’re interested in starting early, sign up to be first through the door when we open it up to early testers. If you’d like to learn more, read it about it on the Go Blog.

In addition, when we take App Engine out of preview in the second half of this year, we will provide a 99.95% uptime service level agreement, operational and developer support, offline billing, and a new Terms of Service agreement geared towards businesses. We will also introduce a new pricing structure for App Engine based on more transparent usage-based pricing.

We’re announcing these features and pricing changes in advance so that our customers have time to review them. Adding business features will help App Engine meet a broader set of needs and the new, more transparent pricing model will help customers better align their App Engine investment with their business goals. Learn more about these changes on the App Engine blog.

Finally, we are really excited to graduate from Preview and we want to let everyone know how much we appreciate the support, commitment, and feedback we have received. We look forward to working with you for years to come.


Greg D'Alesandre is now the Senior Product Manager for App Engine after coming back from riding the Google Wave in Sydney. And he's obsessed with chocolate, no, seriously, obsessed.

Posted by Scott Knaster, Editor