MANDY WAITE: Hi everyone My name is Mandy Waite from Developer Advocate for the Google Cloud platform Welcome to Google Developers Live FRED SAUER: Welcome indeed I’m Fred Sauer, also Developer Advocate on the Cloud platform Today we have a number of questions that you’ve already submitted that we want to answer But first, Mandy, you have a little presentation for us? Kind of give some of the viewers who are not as familiar with App Engine an overview, and then we’ll get right into your questions MANDY WAITE: Yes, so most people who will be on the live stream are actually people who have signed up for the App Engine startup pack So welcome to everybody Thanks for joining the program Hopefully, this will inspire you to use some more of those credits We’re going to look at some of the features of App Engine in this presentation that you may have actually overlooked, you may not have seen, or may not be aware of So we are going to dive straight into presentation Yeah, so we’ve done intros So firstly, the startup value proposition App Engine is a great product, but what’s the real value for startups? So one of the key points– and this kind of stuff is already documented at Cloud.google.com, but we were going to make a big point of these value proposition statements here App Engine is very quick to get started with It allows for very rapid development of applications And that’s important to startups who want to build a minimum viable product in perhaps hours, not days like they used to They want to get their application out to market as quickly as possible And to facilitate that Google App Engine is extremely easy to use We give you all of the tools you need to build your applications, to test, to launch, and then to update

your application’s going forward We also provide a very rich set of APIs and services, which you can build your application upon, leveraging things like mail, XMPP, and other cast services in the background Also, autoscale, people keep talking about autoscale, when I mention App Engine The media scalability almost to most users of App Engine is almost like there’s infinite scale, so you can scale rapidly and as massively as you need to And also one of the final points is that you only have to pay for what you use App Engine is free to use to get started with You get some benefits from sign-up for billing, particularly the credit card, so you guys are all already signed up for billing, but you only pay for the resources that you use as your application grows And here’s some credit, some people that have used App Engine and been successful with it So Khan Academy offer teaching programs online And they’re very pleased with App Engine We use their statements quite often, and these folks they feel that App Engine gives them the ability to spend all of their time developing their application and not worrying about infrastructure Pulse in a very similar position A quote from Greg Bayer there that they could spend time not worrying about the things that they used to have to worry about Google now takes care of those things for them Just to go over very quickly of some of the new features in App Engine We update App Engine pretty much every month, so we’re currently at 1.7.7 And in case you’re still lagging behind a little bit, it’s important to keep up-to-date with new updates And to look at some of the features we’ve introduced recently, in App Engine 1.7.5 we introduced Cloud Endpoints points, which we’ll talk about in more detail; Cloud Messaging support, high memory instances In 1.7.6, a bit of a lesser release, but we introduced Task queue Async Methods, and if you’re developing in Python, we finally got a multi-threaded development AppServer In 1.7.7, which was released in April, that was last release, we finally provided Java 7 support it was previously available in SDK SSL support for Python, Maven support for Cloud Endpoints, and also we removed the minimal charge of $2.10 So you no longer have to pay anything until you actually hit your quotas So just going to quickly go over some of those features that you may have actually missed, that maybe not actually be completely apparent You may have started off with App Engine a while back and actually not kept up-to-date with some of the features we’ve introduced Cloud Endpoints is a really important feature for us Basically, what it’s allowed you to do is to develop APIs in a way that’s very similar to the way that Google Developers their APIs You can take an existing application or develop a new application, and annotate your classes and methods within the application, to expose those classes and methods as an API Once you’ve exposed them as an API, you can discover that API, there are various utilities that we provide that allow you to get discovery documents and to build API client libraries for your own API With those libraries you can then go on to build clients for Chrome, for Web applications in general, for Android and for iOS So we thought that Cloud Endpoints is an extremely important feature So if you have interest in building particularly mobile backends– I want to say backends by using the broader sense of the term, the way App Engine uses it– but if you’re interested in building mobile backends, then you should really check out Google Cloud Endpoints Another thing that people sometimes miss is AppStats It’s not turned on by default, you can add it to your application And you can use it for profile in your OPC course You can basically use it to work out where your application is spending time App Engine Sockets This is a highly sought after feature Previously in App Engine you were not able to create sockets of your own to create outbound sockets We’ve introduced that feature It’s enabled for trusted testers at the moment, which means you have to sign up for a program and agree to do some testing and feedback with it, but it is available for you to use today Also we provide a Search API And again some of these services are experimental, which means they are liable to change Cloud Endpoints in currently also experimental They are liable to change from release to release, but they’re worth looking at now Search API is one such feature And that allows you to do Google-like search as across your own application data It support things like custom scoring, which is the ability to create algorithms that score and results they’ve returned And also rich snippeting, which allows you to return

HTML snippets of your search results with the search term highlighted in bold And also supports GeoSearch So if you’ve got the appropriate data on your application database, then you can do GeoSearch across your data And for those of you basically in the European Union, and there are several of you, many of you, in fact, we also support App Engine European Data Centers So this is great for compliance and locality of reducing ping time and suchlike You do have to sign up for paid application for this, but you guys are already signed up for paid– so that’s great– so you can use that today Also, we have a fairly extensive and growing all the time Development Stack We have Maven support, we have Maven archetypes and plug-ins that you can use to deploy your applications Jenkins is also supported in the Cloud for a thing called Cloud Base Online IDA called codenvy, which is also supported in App Engine And we also have the Google plug-in for eclipse, which ties into the SDK And very quickly looking into the storage options Storage is a big thing for application development We have many different options And the ones we’re not going to go into detail today, but you can look them up online and get more details about them But we have Unstructured data support using Google Cloud Storage and Google Drive; Structured data– The App Engine DataStore; and also Relational data by using Google Cloud SQL Very briefly covering those three options I’m not going to actually talk about Drive today Drive is a Cloud product, but we don’t generally talk about it in these particular talks NoSQL DatsStore, so this is the App Engine DataStore– schemaless; it’s for atomic transactions and queries This is really great for internet scale, denormalizable DataSets So really think differently, no joins when you’re using this kind of NoSQL DataStore And also a fairly standard Cloud SQL [INAUDIBLE] is really MySQL in the Cloud It’s very familiar to anybody who’s used MySQL before It’s fully managed and it’s best really for bounded scale We don’t have high levels of scale for Cloud SQL yet And I believe it’s actually out of experimental now And the final offering, last but not least, is Google Cloud Storage And this is really a bucket for storing all types of unstructured data The five things you really want from a DataStore– reliability, durability, speed, low cost and simplicity– are all encapsulated in Google Cloud Storage So you can check those options out online And just a quick mention of Google I/O. Google I/O is going to happen next week If you haven’t got a ticket, it’s unfortunate, but you can stream the events live You can follow up afterwards on YouTube There’s many interesting talks, many talks on the Cloud platform, including the App Engine If you want to be more involved, we also have I/O extended And you can find out more about that from these links here These links will be available on YouTube afterwards if you need to go back have a look at them So we really are going to open up for questions And Fred and I are going to try and bounce some questions back and forth to one another FRED SAUER: This is my favorite part MANDY WAITE: Your favorite part? FRED SAUER: I do MANDY WAITE: Is it? But you’re an App Engine expert, and I’m kind of new to App Engine I may struggle FRED SAUER: Well, listening to the presentation, I think I may dispute that [LAUGHTER] FRED SAUER: Since you have spent a lot of time talking already, do maybe want to throw a question at me and start it that way? MANDY WAITE: OK, yes So we opened up a moderator page on the Google Developer Live site So we have several questions from there, we pulled some Well, we’ll maybe also get a chance to look at some of them afterwards, but we have like 20 old questions we can go through FRED SAUER: OK MANDY WAITE: So one of them, the first one, was I want to develop a restful web service using Google App Engine that’s consumed by my web application when in a different domain How do I restrict the calls only to my web application? What are the best practices? FRED SAUER: OK So this is a question of cross domain, or what the standards like to refer to as cross-origin requests So if you look on the W3C web page, you’ll find something called CORS C-O-R-S and it stands for cross-origin request source? I forget the abbreviation, I use it too much MANDY WAITE: Just call it CORS, right? FRED SAUER: Cross-origin request sharing, I believe what the actual abbreviation is And basically, this is a mechanism whereby servers and browsers agree on how to communicate the domain of the

website that is making a request to the server, and from the server to either grant access to that content or forbid it And what you do is when you set up simple resources, like even images in an image tag, or when you make an XML HTTP request, you can specify that that request is a cross-origin request– so it’s headed for a domain other than the one that the website’s running on So maybe your website’s on mygreatapp.com and the Backend service that you’re connecting to, an App Engine, is actually on mygreatbackendservice.com And you can do that, but you have to set this extra flag in your HTML in your JavaScript to make the cross-origin request And then on the server, for a get request there’ll be an extra HTTP header coming in as the origin header And this is an indication to the server that this request came from another origin And right then you can make the decision in your request handler whether or not you want to allow that request And you do that by sending another response back, which is the access control cross-origin allow And you can specify the domains that are allowed If you make a post request or a put request, than the browser will do something even smarter than that; it will send what’s called the preflight request, and will send a request in advance and get permission before it actually does the submission So the short answer to this question is are there existing senders to do this? It’s very easy to do Just look up CORS, C-O-R-S, and implement the details there And you can do that in your request handler in your HTML app, and you should be good to go MANDY WAITE: Excellent, well You’ve got one for me, right? FRED SAUER: I do I do MANDY WAITE: Is it an easy one? FRED SAUER: Hopefully Come up with a shorter answer then than mine Maybe we should have started with an easier one Let’s see, so how viable or recommended would it be to build a back in service for mobile apps around Google Cloud Endpoints? MANDY WAITE: All right, OK, so I think really that’s probably the big use case for Cloud Endpoints, built-in mobile Backends And when we say Backends, again, we’re talking about the general sense of it rather than the App Engine sense of Backends FRED SAUER: Yeah MANDY WAITE: If you want to build an application that will provide resources and services that are consumed by mobile devices, then Cloud Endpoints will do that for you out of the box I think pretty much we covered most of that in the slides But the documentation is pretty extensive, both for Java and for Python So if you want to find out more about how to use Cloud Endpoints, you can check that out And there was also Google I/O Talk last year, wasn’t there, the one that was long? FRED SAUER: There was, yes MANDY WAITE: So you can actually get a lot of information from looking at the Google I/O sessions from the previous years FRED SAUER: Very good MANDY WAITE: Yeah? FRED SAUER: All right, your turn MANDY WAITE: My turn? OK, so last week– maybe that was last week, or it could have been written last week Last week I had to rename an entity-kind name for Google App Engine Java I had to write one-time code in my application to duplicate each entity to the new kind name, delete all entities, then remove the one-time code, and re-apply the application Is there an easier way of doing this? FRED SAUER: I think this is a little bit of a longer question But I think the person who asked it is actually on the right track and how to do this So first, let me contrast this with how you would do things in a relational data world, where you have a fixed schema for your database So if you have, for example, a table in a relational database that’s called customer And you decide, for whatever reason, you want to restructure that, and you want to call it customer 2 What you would actually have to do is you would have to declare a maintenance window, you’d have to take the website down, and you would then perform the database maintenance You would rename that table, which locks the entire table and all data in it You would modify the code in the application server, you’d bring everything back up And maybe half an hour later, maybe a few hours later, you’d be up and running And that was maybe acceptable a number of years ago, when website regularly went under maintenance, and a lot of systems were just 8 to 5 But now we live in a world, where everyone is connected to the internet, it’s always 5 o’clock somewhere Everyone needs to get to your site all the time And so what you really want is a system, where you can make changes while the system’s up and running And part of the question actually has the answer here and how you do this So the specific question was, how do I rename an entity? But you can generalize the question, what if you wanted to add new required properties to particular entity in the DataStore? Or you wanted to restructure, you’re going from three separate entities and you’re combining them into this new

data structure, where you put everything in one entity And these are common refactoring that will happen with your application And so the way to do this is to think about this life system that’s always handling transactions And it’s actually a three-step process In the first step, what you want to do is change your application, so that any time that you write out an entity, you’re writing it out in the new format or with the new entity name, or with the new properties, the new constraints But at the same time, that every time you do a read, you first look for the new kind of entity, and if you don’t find if, you look for the old kind And you deploy that And as soon as you deploy that, your application will slowly begin migrating data Any user that logs in and touches their data, those entities, those rows in the DataStore will begin to get migrated And now, actually, you’re in a perfectly happy state Your application will continue to function There’s a little bit of extra read overhead for people as they’re migrating, but you now have all the time in the world to make the full migration happen You’re still serving live traffic And now, depending on what you want to do, you can either say, we’ll just leave it like that But oftentimes that extra code there is complexity you don’t want, and you might want to get rid of that And so what you do is then you run a, what we call a MapReduce, which essentially iterate over all the entities of the old kind in the DataStore And you just touch each one You can do that in ApplicationLogic And as soon as you touch each one, it gets migrated to the new structure And you can take an hour to do that, a day, you take weeks or months– really, however long it takes And once everything’s been migrated, then you go back into your application and remove that extra code in the read handler And your apps migrated And many developers have many of these migrations in place at once, they might introduce a few migrations incrementally, and then maybe once a quarter do a clean up and get rid of some of the code It’s not the case that you have to migrate everything in a short window Think about doing things live in real time MANDY WAITE: Wow OK That’s cool. you have one for me? FRED SAUER: Yes All right, here’s a developer who is a little bit conscious about the bill at the end of the day He says, each day I fight to really save every penny on App Engine– MANDY WAITE: We all FRED SAUER: We all do How could you help us better profile our applications, Appstats– it’s something you actually mentioned– he thinks is a little bit hard to use And what’s the best way to tune your application? MANDY WAITE: An interesting question, OK So, yeah, we mentioned Appstats earlier I think we’ve plenty discussed it before We think the Appstats is probably the way to go It’s built into the platform already, no external tool is needed So, it’s there really to go It would only profile your own PCs It won’t tell you where all of the hot spots that need to be cleared up, but it will definitely give you a really good indication about what your application is spending its time doing, particularly in terms of RPC calls– calls of the DataStore that includes as well It’s very easy to get up and running as well So we’re not sure if it’s difficult for you to actually get up and running, and using it, or if it’s difficult for you to actually get anything meaningful from it But generally, it’s fairly easy for both Java and Python to get up and running with Appstats To get access to the console– they provide a custom console you can have access to, which can actually include your basic admin console Once you have that available to you, you can look at your own PCs, you can look at each one of your requests, open up the request and then drill down into the request to see where you’re spending your time And even then for Python– I’m not sure if this is available for Java, but with Python you can actually look at the code that actually generated RPC call FRED SAUER: Yeah, in Python you have stock traces, that’s right MANDY WAITE: Exactly, yeah, so with Python you can actually try showing to where the code is– and it’s actually called longer RPC calls– and actually modify them, redeploy the application, see what result you got So that’s really cool So I think we recommend using Appstats, definitely for RPCs I’m not really familiar with any other profilers for App Engine, although I did briefly, the Khan Academy actually produced a profiler Ben Kamens’ Maven So hopefully worth looking into FRED SAUER: Yeah, check out Ben Kamens’ blog He does have a Python profiler with some nice UI widgets that tell you about the loading performance of your site To finish up on the Appstats, I think, what some of our users sometimes run into is they feel like all the information that Appstats provide is really overwhelming And it’s good maybe to take a step back And one of the things that Appstats provides that I really enjoy looking at is there’s these timeline graphs that give you a breakdown of the RPCs visually

And 99% of the time when I’m using Appstats, I only look at the graphs There’s a whole page though of stack traces and debugging information, and data that’s been sent back and forth Almost everything related to performance of your application you can find just by looking at the graph You can see whether you have DataStore reads that are staircased all sequentially behind one another, whether they’re happening in parallel You can see that you’re making them in cache call and then a DataStore call, which tells you there’s a in cache miss, right? MANDY WAITE: Right FRED SAUER: So really just getting comfortable with those graphs, I think, will provide most of value with very little effort MANDY WAITE: OK Would you recommend it for running on production as well? FRED SAUER: I do, but depending on the traffic of your site if you only have a few requests per second, then, yeah, sure, run it If you’re doing a lot of traffic, what we recommend is that you either deploy another version in production that you hit, so you can say version dot your app ID dot [INAUDIBLE] dot com and access that directly And that way you don’t have all the data from all the requests from all your users piling on top of each other, but you can very carefully just poke at your application, say, I’m going to do this one request here and then I want to see the Appstats profile MANDY WAITE: Yeah FRED SAUER: We’ve even had people with mobile clients They’ll have a couple users that report performance problems, and they’ll change their application to send down a ping to those users, and say, OK, this mobile client should connect to a different backend name And then those users are isolated on their own version, where you can do diagnostics, maybe you can enable some traces, you can enable Appstats And then, when everything’s all [INAUDIBLE], your tell the clients to go back to the original version MANDY WAITE: OK FRED SAUER: So, yeah, a lot of ways to utilize that tool MANDY WAITE: All right, definitely, OK Well, that’s good Hopefully, that answered the question So one for you FRED SAUER: OK MANDY WAITE: Any suggestions on how to simply move data from the live server to development server, and visa versa? FRED SAUER: Sure We have in the App Engine admin console there’s a data store backup feature, and it allows you to create a snapshot of your database And those snapshots can be placed into your Google Cloud Storage, so it’s just a very large file that you can then download, you can store it offline But you can also restore that back up to another app ID And you can select actually which kinds get backed up So that’s a great way to copy data between instances If you have like production and QA environment, and you need to get another snapshot of data It’s pretty common Or if you have been working really hard for months in the development server, and you have all this configuration data, maybe you want to just take those kinds, grab a backup of that, and restore that, play that back essentially on onto the production server MANDY WAITE: Well, sounds easy OK FRED SAUER: Yeah? MANDY WAITE: OK Excellent FRED SAUER: OK, for you let’s see I need a testing strategy for App Engine applications as the SDK doesn’t implement all the services exactly the way that the production environment does, do you have any insight about how to test your applications taking into account some of those differences? MANDY WAITE: Yeah, so I guess testing, there’s a lot of ways to do testing, there’s a lot of levels you can do testing– you can do unit testing, functional testing, integration testing Development testing is pretty important of App Engine, you really probably don’t want to deploy an untested version of an application and replace the current version that’s working really well with one that’s not been tested, particularly extensively So, as always, when testing in the development environment, it’s probably important to mock services that you can’t actually access directly FRED SAUER: Sure MANDY WAITE: So in this case, if there are any shortfalls in what the development server offers and what App Engine offers itself, then you probably want to mock that FRED SAUER: OK MANDY WAITE: So that would be the same with accessing another API that you don’t want to access from development You may just mock that service up, so you can actually get reasonable results from making course of that service Then you’d probably do the same with App Engine as well if any App Engine services are missing And beyond that, really there’s lots of different options for testing Once you’ve deployed the application, you may, as Fred’s already mentioned, you may deploy a different app version, and then you may use traffic splitting So App Engine has this feature of traffic splitting, that allows you to incrementally direct traffic towards new versions of your app So you can actually test it in isolation to a degree, You’re pushing a small amount of traffic to it initially, and then slowly migrating all of the traffic over as you get more confident in the application’s performance And again also that other kind of testing, low testing and suchlike You don’t really want to do load testing on a production

applications You would probably need to build a different version of your application, then to pull it out, and undo your testing on that particular version of your application You’re also likely to need data as well, so you need to probably have a representative set of sample data that you can deploy your application If you have an empty DataStore, it’s not really going to give you the exactly same kind of results that your production application would do, so you probably want to divine a set of data that you can actually populate into your application initially to do the testing mode FRED SAUER: Yeah Those are very, very, very good points The one thing that bites people a lot of times when they’re testing in a different application ID, or even on the production application before they launch is people will not use representative workloads MANDY WAITE: Right FRED SAUER: So they’ll instead of creating say 1,000 unique accounts, which is sometimes tricky to do, they’ll say, OK, we’ll take 10 accounts and we’ll have each one log in 100 times running concurrently And those access patterns actually change the way the data is utilizing, and you don’t get very representative results there So you already talked about making sure that you have some good sample data there, making sure you have good representative requests, and users logging in It’s just as important In any case, as much as you can take the production environment that you expect and replicate it, the closer you get, the closer your results are going to match, obviously MANDY WAITE: OK, OK So you can use tools like Selenium and JMeter, those kind of things for low testing FRED SAUER: Absolutely MANDY WAITE: The same kind of tools you would use for any kind of workload web application to generate tests And they generate representative workloads, believe me FRED SAUER: They do Just watch out that you’re not testing from one machine that just doesn’t have a big enough network connection MANDY WAITE: Yeah, so don’t talk to me about client side, because if your client can’t handle the load, you’re never going to actually load on the server the way you think you are So you need to make sure you can actually generate the load correctly, you’re not falling over lack of client resources as well when you’re testing FRED SAUER: I guess you can always get a few instances on Compute Engine– MANDY WAITE: Ah FRED SAUER: There’s some capacity there MANDY WAITE: That’s interesting there Let’s go there FRED SAUER: Nice big pipe of data Let’s see, where did we leave off? It’s your turn MANDY WAITE: I’ve got one for you, yeah Could you provide us with state-of-the-art database model designs? OK, that’s a big ask Things like CRM database model, a bookshelf, an eBay kind of application FRED SAUER: OK, it sounds to me like the person asking the question is looking for sample model, sample applications And I hear the question behind the question, which is how do I go about modeling an e-commerce site, a CRM site in this non-relational, NoSQL world I want to build an application that can handle tremendous scale, but you’ve swapped the tools out from me I’m used to doing development on a SQL database, relational, I know how that works, but I know that that doesn’t scale So help me switch MANDY WAITE: Right FRED SAUER: I think it’s difficult to come up with samples that fit every single vertical But there are some general things that we can say about the way you do data modeling And really it’s sort of changing some of your habits and some of the things that you were taught maybe from the beginning about working with relational databases So one of the first tenets when you’re building a relational database model is everything needs to be normalized No data duplicated It’s typical in an order entry system to have 5, 6,7, 10, 12 different tables, that every time you want to do a query and out what the orders are, you have to join all these tables at run time and select your results So you might have a table that has items in it, and it has item description, item price, and some other information; then you have an order lines, which connects to an header, you have order line details, it goes on and on And this is a really good way of allowing you at run time the flexibility to really join any amount of data with any amount of data, which is what relational databases were built for But in a very scalable world, what you want is actually that if you and I both make a request to our website and we’re both interacting with the service, what we want is for your queries and my queries to essentially be able to be handled by different parts of the infrastructure it’s a distributed platform And the way to do that is to isolate your data from my data And that means what we don’t want is for if I’m pulling up a list of my orders and you’re pulling up a list of your orders, we don’t want to both be going to the same table and

running into each other We also don’t want to take a lot of time at run time collecting the data In a distributed system, what you like to do is amortize the cost spread out the cost and do that the cost at right time So when we write the data how we might do some deduplication, so what you would call denormalization So it’s quite common in App Engine apps for entities to have properties with string values So rather than have an item key on your item details page, you would actually have the actual item code or the actual description If you’re putting together, let’s say, countries You have a client that stores addresses and you need countries In App Engine you would actually just put the country code, or even the full country name in every single entity And you should resist the urge to say, oh, I don’t want the duplication of data, I don’t want to pay for the extra storage What you’re really doing is you’re separating your data and my data, and making it very efficient to handle very large number of requests There’s more best practices, but the first one that you want to think about is denormalization is not a bad thing That’s actually something you should really just embrace And just pick two users and imagine that they’re both doing the same thing in your application How can you make it so that the entities that you’re touching and the entities that I’m touching are different, because as soon as they’re separate from each other, then the Google App Engine infrastructure, the DataStore, can put those entities on different physical hardware, different machines And now we can have a third user, a fourth user, a fifth user We can have 100,000 users, it really doesn’t matter how big we scale up, because each user is accessing a different part of the system MANDY WAITE: Wow FRED SAUER: So there’s a lot to grasp if you’re coming from a SQL environment, but once you play around with it for a while, it’s really freeing to say, I really don’t care how many users show up on my doorstep tomorrow, I can handle 1,000 users, a million users, a billion users It’s really independent of scale And that’s what the App Engine DataStore provides is performance that’s independent of the amount of data that’s stored MANDY WAITE: Yeah Makes sense It’s really good advice FRED SAUER: All right, so, sorry for the long answer there MANDY WAITE: No, I think it’s the kind of question that deserves a long answer FRED SAUER: Yeah, it’s a common question, so it’s fair to spend a little bit of time– MANDY WAITE: A lot of people moving from the old traditional relational database to the NoSQL type one FRED SAUER: Yeah And we should say, is if you do want a relational system, you have a limited number of users You’re building a corporate application You only know your need for the existing performance You can use Google Cloud SQL– it’s MySQL database, runs in the Cloud, managed for you, no headaches And you get all the traditional kind of performance trade offs and benefits It’s still an option MANDY WAITE: Yeah FRED SAUER: All right, next question for you MANDY WAITE: OK FRED SAUER: Any optimization recommendations for e-commerce website with two million visits per month, running on App Engine Python MANDY WAITE: Well, OK, how long have we got? Optimization recommendation– I guess what we want to optimize for is probably important thing here Do we want to for performance, for load, for latency? The cost– and cost is an important consideration, when it comes to Cloud applications as well balances cost and performances Massively important So I think what I’m going to step back on this one a little bit and say, the best thing to do is to read the resources that are available in the documentation So we have a couple of really, really good pages on the App Engine documentation site One is called Managing resources– FRED SAUER: Yes, my favorite one MANDY WAITE: That’s Fred’s favorite one Whenever I ask him a question, he has have you read managing resources? So I go off and read managing resources, and I’ll find the answer So managing resources is great It tells you have to manage your instances, manage bandwidth, manage concurrency and those kind of things And so that is an excellent resource There’s also a performance-related resource on the same site I think they’re in the same section, probably one leads into the other And I will give you hints on performance Often optimization and performance are kind of related, very married together in a lot of ways, so you can take those tips as one big set of tips And you’ll fine really good, good information about how to optimize your application If it’s about cost again, I think also managing resources also talks about how to look at your quotas and things like an where you’re incurring your costs And also that becoming back to Appstats So Appstats is another really good essential tool for looking at the way you’re spending your time in your application So if you’re concerned about latency, having taken too long to respond, so you go and use this, then you can do some analysis with Appstats to find that way you’re spending time, and maybe tune your application to spend less time doing those RPCs FRED SAUER: OK MANDY WAITE: Does it seem good?

FRED SAUER: Yeah I love the managing resources article If you just do a web search for App Engine managing resources, you’d probably hit that article in our Docs And think of it as a checklist It doesn’t tell you exactly of everything to do, but it’s a checklist– think about this, think about this– and you go through each one and you know your replication best I.e. can tune it If you just want the what’s the one thing I need to do that I may not be doing that’s going to give me the biggest benefit? I say, turn on Memcache And some of you can get that for free If you’re using in Python the NDBAPI, or in Java, if you’re using Objective, which is a third party [INAUDIBLE] layer, they both have Memcache built in So any time you write or read from the DataStore, it’s checking Memcache That can save you a lot of DataStore operations, and it won’t cost you a single line of code MANDY WAITE: Wow, fantastic So use Memcache FRED SAUER: Use Memcache If you’re not using Memcache, use Memcache MANDY WAITE: I was about to suggest you use Mamcache OK, so I’ve got one for you Oh, another performance So what is the performance difference between GAE Python, GAE Java, and GAE Go? There are three different run times FRED SAUER: OK Fair question This is actually one that comes up a lot It’s a question that I tried first not answer, and then I’m happy to give an answer But I’d like to start out and say there are subtle differences between the three different runtime environments Python, if you write any application in Python versus Java, versus Go, there are some things that work better in one language versus another What’s probably much more important for your application For your service that you are running is what your developers know and where they’re going to be productive So if you have a Python shop, everyone knows Python really well, and there’s one or two guys that know a little bit of Java; even if Java were slightly better for your application, you’re going to be much more productive building in Python So you should do that And the same goes the other way around Having said that, if all things were equal, if you hadn’t hired any development staff yet, or you could pick any language, or you just want to learn something new– there are some trade offs to make And maybe you should look at the complexity of the product that you’re building and how long it’s going to last Python is a scripting language, tends to be a little bit more productive for people prototyping, iterating in small teams, working together, let’s you be very agile– you can do things very quickly If on the other hand, you’re building a product, and you have a very large development team, or a very complex code base, you’re doing a lot of refactoring and you could use some help from tools that can refactor code, do static analysis; then Java is probably the language that’s more productive And if you wanted to do something that’s really performance sensitive, you want to build with the stuff that Googlers are building on, I would seriously check out the Go runtime There are some really pleasing results there Having said that, that’s an experimental runtime still, so if you’re building a production app right now, probably Java or Python And pick the one you’ll be in productive MANDY WAITE: So retraining your Java developers, so you [INAUDIBLE] in Go is probably not the best way to get performance? FRED SAUER: Unless that’s what motivates them For some people a new language is the thing that wakes them up in the morning, and more so than a cup of coffee For other people it’s being able to be productive and hit the ground running MANDY WAITE: Excellent OK Well, good advice FRED SAUER: OK Let’s shoot one back at you here Can I pre-populate the DataStore? Is there a front end to add, remove, or update entries in the DataStore MANDY WAITE: OK, so now I’m going to punt this one a little bit, because we discussed this one yesterday, and we talked about some things and some articles that have been published about doing this very thing So I’d like to actually talk to you about that I’ll bounce the question back to you and ask what you feel is the best way to pre-populate FRED SAUER: So, I think the answer depends a little bit If you have some data on your client, like on your desktop, maybe there’s some legacy data that you need to import, some configuration data that you want to do programmatically, the development server of or the SDK does have a little tool for uploading data MANDY WAITE: OK FRED SAUER: That works fine for small scale, and that’s a tool that’s been around for a long time Probably what you want to do if you’re doing anything larger than trivial operations is just create a Cloud Endpoints application version of your app It doesn’t mean you have to build your entire service around it, but maybe this is an administrative gateway, or this is a way that a mobile clients upload data to your application Cloud Endpoints is a great easy way to create an API and

will automatically generate client libraries for Android, for iOS, for HTML5; and from any of those three clients, you can make your calls to the server side If you’re doing Python, a colleague of ours, Danny Hermes has a pretty cool project called Cloud Endpoints Proto DataStore MANDY WAITE: OK FRED SAUER: Or maybe it’s Endpoints Proto DataStore But it’s an application, where you essentially define your Python DataStore models, and then you just swap up the class name that you’re inheriting from, and then all your classes magically turn into Cloud Endpoints And so you can make calls like insert entities into the DataStore, remove them, do queries All really easy So that’s probably the way that I would go And then there’s maybe one more trick up your sleeve, the SDK has a thing called a Remote_API– Remote underscore API MANDY WAITE: OK FRED SAUER: And it’s a way for the development server to essentially proxy its DataStore and then cache request to the production environment So what you do is you deploy a special version of your app with just like a one line config flag, this is enabled Remote_API And then you can run your code locally as if you’re connected to the DataStore, but everything’s been proxy’d to the Cloud for you So that’s another neat trick for migrating data in the Cloud Or it’s a great way to do debugging, you connect to your production or staging environment, and you can interact with it, while you’re in a Python console, you have your real data models, but you’re looking at your real data MANDY WAITE: Excellent OK I would not have been able to come up with that answer So I’ll let you ask me another one now, because I think I have more questions for you than you have for me FRED SAUER: OK Let’s see, startup times for my App Engine Java are pretty high for my applications, around 30 seconds– yeah, that is actually pretty high and Java apps in general– even with reserved instances that we feel are underutilized, we’re still getting warm-up requests Any advice? MANDY WAITE: Yeah, so I think, again, this is something that is going to be covered quite extensively in Google I/O. We have a talk, Matt Stephenson is going to be giving a talk on autoscaling Java So that talk is likely to cover much of that And Matt Stephenson also with one of our other guys from Cloud Solutions, Wally Yau, he wrote an article that was published on the, again, the App Engine Developers documentation site for managing optimization of startup Spring applications And often we find that it’s Spring applications that really take the longest time to start There’s a lot of libraries involved, there’s a lot of scanning of classes and suchlike that goes one automatically with Spring when it starts up And that can lead to quite long low times As I suspect, this is probably likely to be a Spring issue At the same time, there’s also the possibility to go loading libraries that you may not need immediately And you may want to actually look at lazy loading those– loading them in the background– as opposed to actually loading them as you’re instant starts Anything that you can do to minimize the amount of time it takes to start the application is good And really, I think, that’s the key to handling things like a startup requests, things that will cause a new instance to start It’s basically just to minimize the time it takes to load your application initially Only load what you need, lazy load everything Don’t load things you don’t need to load And also make sure if you’re using Spring that you follow the advice in the Google [INAUDIBLE] and in that particular document on the website That will minimize your setup times And you won’t be so worried about startup requests then You can actually loading a new instance, because that one takes that long to do FRED SAUER: Yeah All very good advice So, yeah, definitely check out the Google I/O talk All the talks at I/O are going to be recorded, so we can check those out, talks happening next week So it’s a great resource MANDY WAITE: That’s on Wednesday, by the way Are they live streamed? FRED SAUER: Some of the talks are MANDY WAITE: Some of the talks are live streamed, yeah FRED SAUER: I’m not sure if Matt’s talk is live streamed, but you’ll be able to catch it pretty soon Maybe something– just to say a little bit more general about App Engine Java, especially these Spring apps or applications that have these big frameworks and long loading times So one way to think about this is the way that we’ve been building application servers for many years has been in an environment where you would spin up, say, five application servers And these would be big server that would sit in a rack somewhere And they would take several minutes to boot up, do all the memory checks Machine would load up, it would load the application server software, than would load the specific applications And that whole process could take 10-20 minutes sometimes But then those servers they would run for weeks or months,

or however long they needed to run And in that environment, what you actually want to do is do all the expense work upfront when you’re loading the application, because you know you’re only going to that one time, and then you’re going to have long-lived servers, App Engine tries to really fundamentally deal with applications in a different way When we want to scale up very quickly to handle an increase in traffic to your website, or when we want to scale down, because there’s a little bit less traffic coming in, we really need the ability to quickly spin up new instances of your application, and then spin them back down, at a moment’s notice And so App Engine is optimized around apps that have a very short startup time Ideally, it’s hundreds of milliseconds, maybe it’s a few seconds on the outset When you start getting into a 30-second time frame, that’s really a whole different ballpark Now, if that’s something you want to do and you want to run Spring applications, and you say, I get a lot of benefit out of that, and why can’t I just run an App Engine? You can We actually, I think, a year ago, we introduced some extra knobs on the admin console, specifically for these users What you’ll need to do is go into the admin console, make sure you enable billing, because that exposes all the knobs that we have And then go into your application settings And there’s a couple of settings around Min/Max Idle Instances , Min/Max Pending Request Latency And those allow you to tune the cost performance of your application So if you know you have long loading requests and you just need to keep in a few extra instances around to make sure you can handle those spikes, even though you have long instances, you can actually crank up the min idle instances MANDY WAITE: Right? OK FRED SAUER: On the other hand, if you’re a fast starting app and you’re very cost sensitive, there’s ways to turn it the other direction We recommend everyone start out with the automatic settings, but we do have some special knobs for that And I’m sure Matt in his talk will go into a lot more detail than we did right here, specifically around Java application So that’s definitely something we care about, and we’ll have some advice for you MANDY WAITE: Yeah, that’s great [INAUDIBLE] give excellent advice Oh, you’ve only got one question FRED SAUER: I’ve got one MANDY WAITE: And I’ve got four FRED SAUER: And then we have a few more that came up in a moderator since we looked at these yesterday MANDY WAITE: Well, let me ask you a question FRED SAUER: OK, go for it MANDY WAITE: How popular is Google App Engine for Java? Most examples seem to be reaching for Google App Engine Python FRED SAUER: I hope that’s not to the case as far as the sample mismatch It is true that we launched App Engine Python a year before the Java runtime So in 2008 was Python, and then in 2009 we added the Java runtime, and later we added Go So there may be just a little bit of bias still left in the system, because we originally set out to just have a Python runtime But I think things are fairly evened out If you go to the Google Cloud Platform user on GitHub, there’s many samples there There’s Java and Python samples; documentation, there should be samples in both languages I don’t recall the exact kind of popularity, but I thought it was fairly even, maybe a 60/40 split or something like that, with some percentage are going to go But you shouldn’t feel like Java is an under appreciated language in App Engine, it’s definitely not the case And if you look at the talks at I/O, you’ll then see our focus on Java users and making sure they as just as good experience as Python So pick the language that’s good for you MANDY WAITE: Yeah, definitely, Yeah, absolutely good advice Do you have one for me? FRED SAUER: OK, I do So the App Engine documentation is incomplete, for example, the NDB Docs– NDB is a Python library for the DataStore– remain minimal and really document possible errors Anything you can do about this? Or are there other places I should be looking? MANDY WAITE: Well, I guess really, the one of the standard answers is to go to Stack Overflow You’re going to find documentation, people talk about things undocumented or not so well documented, providing examples and suchlike on Stack Overflow Pretty much everything anybody’s tried to do is going to be documented on Stack Overflow Things that have worked, things that haven’t worked– so you can probably find good answers there If you want to contribute information as well, Stack Overflow is not just for questions It’s also for sharing information as well So if you actually have some useful information– something you’ve written about, something you’ve documented– you can share that on Stack Overflow as well Definitely a good place to put it And it also stops you from being spread around everywhere We could have multiple places where people would go to find different documentation So that kind of ad hoc organization probably is the best place in the Stack Overflow

FRED SAUER: Yeah, I think of the App Engine core documentation is kind of the reference material, the baseline Sometimes there’s Javadocs or Pydocs, two of the methods that may help a little bit further, but some of these, like how do I use this specifically, Stack Overflow is a great place to go And we do sometimes, if we find that there’s a lot of questions around a particular are in Stack Overflow, we do go in and update the documentation, and maybe add some clarification So that’s definitely a place to contribute MANDY WAITE: Yeah, definitely Excellent, OK, so a question for you FRED SAUER: All right MANDY WAITE: OK, I’m going to read this through first, so I know what I’m asking I’ve developed an application– 12 [INAUDIBLE] But today I had difficulties to setup the IT strategy related to the maintenance of the source code Taking into account the amount of leases of Google App Engine platform, do you have any advice? FRED SAUER: If I remember right, I think you were suggesting that the person who asked this question, asked a few more questions And I think you said, it look like they were maybe running Python 2.5 Yeah, so App Engine has a deprecation policy on all of our production services– I believe it’s one year– and that’s our guaranteed to you that we’re not just going to change APIs on you, that when we bump up the next version of the application, your application is going to continue working, the APIs are going to behave the way that they expected Having said that, there are incremental changes that we make Just like the Python languages is– Python 2.5 is not as common any more, it’s Python 2.7 At some point you need to make a switch It should certainly not be a regular burden I have applications that I deployed more than a year ago, and they are still running just fine That’s the experience of most of our users Usually what we find is that developers are making changes because they want to take advantage of new APIs or new capabilities that we’ve launched There’ve been probably a couple of cases, where we’ve had a change in– not a breaking API change, but a new version of a capability that we’ve made available– and that’s from Python 2.5 to 2.7 There are some differences And Python 2.7 provides a lot of great features that weren’t just possible in 2.5, like the ability to handle concurrent requests, there’s many language preferences We also had to change from the Master/Slave DataStore to the High Replication DataStore, which has huge advantages The Master/Slave DataStore had regular maintenance periods, there are still a few users on that But I believe the majority of users are now on the High Replication DataStore Just a much better experience When you commit your data, it gets committed to multiple data centers synchronously When a data center goes down, you don’t even notice it And that sort of improvement is sometimes worth going through the effort and making sure that the way you’re executing queries is still compatible with the new version, but you have plenty of time to make that change on your schedule So what we find that developers do is they use one version of the SDK, they’re developing locally, they go through a couple of releases on their own And then there’s a break, maybe once a quarter or so, they say you know what, let’s catch up to the latest features Let’s see what we can incorporate that we haven’t yet And then they’ll bump up the version And so you should naturally flow along with the App Engine releases MANDY WAITE: OK It’s important to also stress what we mentioned earlier Experimental features are always likely to change from one release to another So if you are using an experimental feature, you are probably getting an enormous benefit just from using that feature But you may have to adjust your code when we release a new version, just to keep up-to-speed with us Hopefully, that’s not a big impact, though– not something that causes huge– FRED SAUER: No, it’s generally a polish around the edges when it’s in the experimental Or we’re still trying to collect feedback from the community, find out new use cases; Maybe there’s ways that we can improve the API a little bit And so you’ve seen that, for example, with the search API that from the very early days to now, there’s a couple of changes But generally things have just gotten better, and all of our users have been quite happy with that MANDY WAITE: All right, excellent, OK So we’ve got two questions here You can ask me one if you want FRED SAUER: OK, well I have a couple here too I can ask you MANDY WAITE: Oh, I see, yeah, we can do it You’re putting me on the spot now FRED SAUER: I am Let’s see Please share tips on rate limiting requests without having to hit the DataStore

Memcache-based counter could work, but item expiry time is a little bit unpredictable How would you go about rate limiting individual users? MANDY WAITE: Interesting, I don’t know That’s probably the kind of thing that I have not actually run across yet So I’m going to punt that one to you as well FRED SAUER: OK, well I think the person here suggested a really good place to start, which is a Memcache-based counter Memcache is using– at least recently used– an LRU-based cache And that means that the users that are most heavily accessing your application, those Memcache keys are more likely to actually be in the cache And those that haven’t been around a while, they’re further down the list, and those will expire So you sort of naturally get the keys that you care about most recent memory If you’re looking at maybe even more aggressive throttling, where maybe you have some abusive users, and you really want to cap that behavior down, you can take it up a notch and do some throttling within the instance So you could introduce a data structure, a global static variable, in your code, that for each App Engine instance keeps track of the hot users or users recently seen, and maybe throttles those individually And then if there’s a miss there, then you go to Memcache And what you’re doing is you’re just taking up another tier You have at the bottom you have the DataStore, where there’s transactions happening And there’s this limit of one transaction per second that you want to deal with per entity group Then Memcache, you can go much faster And then in the Instance cache, you can go even faster, because you’re just in memory There’s no cross-network call to be made MANDY WAITE: OK FRED SAUER: So, yeah, I think Memcache is a great place to get started and you have to use Instance caching if you need to there Swell Right, you had another one, you said? MANDY WAITE: I do, indeed, yeah Last one– last one from the list We have a few more over there So for GAE Java, there are detailed documentation for low-level DataStore API However, fewer and incomplete documentation for how to map entity relationships with JDO and JPA The data nucleus is detailed, however, there’s no easy way to tell if GAE supports a feature So, there’s a question in there somewhere FRED SAUER: Yeah, so the question is really, I think, around the giving to this the full details and capabilities of JDO and JPA on App Engine Maybe I should put this a little bit in context When we launched the Java runtime on App Engine, we looked at the act of frameworks and an abstraction that people were using, JDO and JPA were very popular And so we did a lot of work to make sure that those were properly supported in App Engine We also felt like users really benefited from SQL like language, and so we launched GQL, as a way of accessing the DataStore And what we found over time, actually, is that developers after a little while feel much more productive when they’re working much closer with low level APIs of App Engine, or some abstraction that sits on top of those low level APIs So in the case of Python, we had the low-level Python DataStore We’ve now iterative on that and we have NDB, the new database, which is the way that we wish we had written the first time with Memcache built-in, and just usability is a lot better On the Java side, we’ve actually had the community step up and produce an [INAUDIBLE] framework called objectify And there’s others out there There’s a developer in Japan who created something called slim 3, it’s another abstraction layer for Java And they do a really good job of obstructing the way of the DataStore a little bit, but still staying very close to the native performance characteristics doing things the right way Objectify, I know, also has Memcache built-in And it just lets you work with POJOs– so plain old Java objects So you have a Java object with a bunch properties– getters and setters– you can just take that object and push it in the DataStore and get it right back out And that feels a lot more natural working in App Engine than in JDO and JPA And so the way to look at those technologies right now is if you’re a big JDO and JPA fan, or you just use those in your environment and you’d like to, please continue to use them, they are there for you

But if you’re new to App Engine or you’re at all unfamiliar about JDO and JPA, by all means, use either the low level APIs or something like Objectify in Java, because you’ll just have a much better experience And those were written from the ground up for App Engine Whereas JDO and JPA were written in a world where SQLs came And it’s a little bit of an unnatural fit We got that square peg into the round hole, but we had to use the hammer a little bit, and so the corners are a little bit rounded off MANDY WAITE: OK I come from a Java EE background, so I used to have a very big hammer FRED SAUER: I’m just visualizing the big hammer Let’s see, a question here for you At the rate at which you can write the same entity group is limited to one entity group of write per second, and this developer is writing, it seems really low If you’re imagining a Facebook applications with 200,000 daily active users, which means something like 20,000 concurrent users on peak So they’re kind of contrasting this They say, well, on the one hand I have 20,000 concurrent users all making requests at the very same moment, and you’re telling me one per second? Where is the disconnect? MANDY WAITE: I think here depends on what your entity group was actually developed to represent and how extensively you’ve modeled it and the way you’ve modeled your data If your entity group is really specific to the particular user that’s making the call, then you won’t really have to worry about it One write per second will be perfectly adequate But if you kind of sprawled it a bit, and the entity intergroup touches multiple users, then you’re likely to have some contention in that entity group And so you have multiple users banging away at the same time So really it’s best practices of modeling your entities that will actually avoid that kind of issue So if you have 20,000 concurrent requests, they’re likely to be accessing 20,000 different entity groups You wouldn’t have an issue FRED SAUER: And that should be fine If you want to do two million concurrent users on two million different entity groups, that’s absolutely fine MANDY WAITE: Absolutely FRED SAUER: So think about entity groups as, for the most part, your unit of transactionality So If you need some data related to a given user, like a user and their achievements, for example, and you have those stored in three or four different entities, you can put them all in one entity group, and then the App Engine DataStore will make sure that you can only have one transaction in flight at a time for that entity group So generally what we see is that each user becomes an entity group, or each order in an order entry system, each customer in a CRM system becomes an entity group We actually also have– we didn’t have this initially, but now we introduced this again about a year ago– something called cross-entity group transactions, or XG transactions And that allows you to transact up to five different entity groups in a single transaction So it used to be the case that before we had XG transactions, there was a little bit of this trade off between I want to make my entity groups bigger, because I want to do transactions, but I need to make them smaller to have the right throughput And that was sometimes a challenge There were in fact libraries that sprung up to try to figure out how to– the classic example is I’ve two bank accounts I want to move $10 from this bank account to that bank account, I need to do that within a transaction And if I deduct 10 here and then add 10 there, and something goes wrong in the middle, the $10 had disappeared Or if I add $10 first, and then remove $10, I’ve created $10 out of nothing MANDY WAITE: I like that one FRED SAUER: Well, let’s do that one With cross-entity group transactions, that’s no longer a problem, you can actually in a single transaction make that change So really, I think, this is all about just structuring your data store in such a way that you do no more than one break per second Another classic way that people run into this is they’ll do something like they’ll create a site counter They want to know how many visitors came to the website MANDY WAITE: Exactly FRED SAUER: Favorite example, right? And do every time a user comes onto the website, they increment the counter And this is what you would do in a SQL Database, you would increment a particular row in the DataStore and the problem is only one person at a time can update that one record, because that one record is on disc somewhere, and there’s some server responsible for it And you can only touch that record one transaction at a time And that’s really limiting for the number of things you can count And so typical strategy that you use for the App Engine DataStore, is to create something called a charting counter, charted counter, where you partition the counter into multiple counters So instead of, saying, say, one counter, you split it out

and say, OK, let’s make five counters or 50 Some number, and it’s configurable And now every time someone comes to the website, I’m going to at random pick a number from one to five, and then I’m going to update that counter So, let’s say, it’s counter three this time, the next visitor comes in, oh, it’s again, the counter three Oh no, it’s counter two And you say, well, that’s weird, because now your total patriot count is split up around five different counters But that’s not a big deal, because you can easily select five numbers and add them together This is a very easy task for a computer to do But by doing so, you’ve just increased the throughput of your web counter fivefold If you need 50 transactions per second, you go 50 fold, plus a little bit buffer So maybe you go 60, or something like that It’s very easy to chart your counter out as far as you need to go MANDY WAITE: There’s actually examples on the website, isn’t there? FRED SAUER: Yeah, it’s an article So with a little bit of data modeling you can do as many concurrent users as you like And, hopefully, you do a lot more than 20,000 But 20,00o is pretty awesome I wish my website had– MANDY WAITE: 20,000 is pretty very good, yeah Well, I guess the message here really is avoid shared meterable stay I love shared meterable stay as a saying Every time you share, state is meterable You’re going to run into problems with concurrently So just keep it isolated And– FRED SAUER: If do that, you’re going to have a bad time MANDY WAITE: You’re going to have a bad time Absolutely FRED SAUER: Do you want to ask me these ones? MANDY WAITE: Definitely, yeah This one, I think everybody is interested in This person is asking the question, what are 10 different definitions of the frontend versus the backend instance? And I’m still not very clear what the difference is between frontends and backends Is that something you can explain to everybody? FRED SAUER: Yeah, so, actually, you mentioned this in the introduction, sort of the two different notions of a backend You talked about mobile backends for Cloud Endpoints And then App Engine has a specific definition of what the frontend is and what the backend is And, frankly, other customers have said that’s confusing And we found it a little bit limiting in what developers want to do with their applications And so if you have been actually reading through the release notes carefully, you’ll notice that we’ve already started talking about a new way of classifying components in an application, called servers And what we’re going to do is we’re going to get rid of the two special cases the frontends that do things one way, and the backends, who do things slightly differently And we’re just going to say, you can have as many different types of servers as you want You can call them frontends and backends Maybe one of them is your mobile API, and one is your admin API, and the third one is something experimental Maybe you have a component that’s text message processing, another component that’s real time interactive stuff, and the third one that’s reports Every application is different But when you get beyond a trivial application, you often want to logically split your application into pieces, or you may want to deploy the different pieces separately Like your mobile backend that’s handling all your mobile requests, maybe you have a new version for that And with this we’ll allow you to see separate traffic graphs for each of your server versions– your logs will be separated out So your mobile backend logs and your static image logs can be separate So all of that’s coming Just check out the release notes, you’ll see that And just forgive us for calling them frontends and backends, we’ll make that go away You have an application, you split it in logical pieces, and we’ll allow you to configure the different pieces the way you want, whether you want different instance sizes, different logs, et cetera MANDY WAITE: So things like what if [INAUDIBLE] has currently autoscaling, that would just be a property of a server instance– of a type of server FRED SAUER: Exactly So you say, if I have a mobile backend, autoscaling on I have this batch backend, where I’m only going to do certain administrative tasks I actually only want one instance of the server running, and it’s just going to be crawling some database or doing some background work And you can have variations in between MANDY WAITE: Excellent, OK Sounds good FRED SAUER: So we’re making it easier and we’re simplifying the terminology MANDY WAITE: Fantastic Thank you So you did explain it FRED SAUER: I hope so All right, question here What are the best practices for migrating from Python DB to the Python NDBAPI? MANDY WAITE: OK, so I’ve got a confession to make here FRED SAUER: OK MANDY WAITE: I’m a Java girl I come from a Java background So I don’t really deal with a huge amount with NDB I’ve actually looked through the documentation, it looks really cool, but I wouldn’t know where to start FRED SAUER: OK, well, so, I have been calling myself a Java guy, but in the last year and a half or so, I have to

confess, I’ve been becoming somewhat of a Python fan And I’ve been taking my own projects and migrating them So I think I can say maybe a few words about this First thing is you don’t have to migrate all at ones If you have an application, you have many different Python classes, and you’ve models all over the place, you can actually migrate a class follow the time, you can migrate a model at a time, you can intermix the two APIs And the good news is that along the way you’ll probably, if you’ve done things around Memcache caching, there’s a lot of code you can delete, because NDB just does things for you The syntax between the two is a little bit different So one of the first things you might do is maybe go around hunting for there’s a cheat sheet I’ve seeing that has the DB way and the NBD way And that in the beginning is very good for just like mapping Like, oh I need to change this around and do this But you’ll find that they’re very, very similar And like I said, just do one model at a time, one class at a time; test it, make sure it works And then keep going And maybe what you do is you start out with any new models that you create, do them in NDB, so you start to get really familiar with it, and then migrate your old stuff But like with any real-time application, where you’re always serving requests, think about ways of doing things incrementally, rather than in one big step I don’t think there is really anything, any other tips for people migrating MANDY WAITE: What about people getting started? So like when you’re starting out developing an App Engine application using Pythons, is it immediately obvious in the documentation suchlike when you’re developing with DB and when you’re developing with NDB? Is it easy to get pushed on one path? FRED SAUER: You can accidentally flip between the two– both are in the left nav of the documentation, but it’s very clear in your code, because in one case you’re importing from Google App Engine EXTDB; and the other place you’re doing NDB So there’s no confusion there MANDY WAITE: OK FRED SAUER: But, yeah, I do find, when I’m clicking around the documentation, I sometimes end up looking in the wrong one You’ll get through that MANDY WAITE: So I guess if people are reading books on App Engine development, Python-based, they may be looking at older examples that use DB, and probably worth spending some time, investing some time to actually look at the differences in how you would do that in NDB, rather than actually just writing up the samples in the DB code FRED SAUER: Absolutely, yeah, if you’re building anything new, you should only be using NDB The author who wrote both basically he rewrote it and said this is how it should have been done the first So I can see a future where DB is either deprecated or tucked away in the documentation for very few to find And NDB is just the way you should be writing applications And we’ve actually just going through a process with our sample applications So if you look on again GitHub on the Google Cloud Platform, hopefully right now all the samples have been updated to NDB So if you want good samples of best practices and how to do that, definitely go to our GitHub page MANDY WAITE: Excellent OK FRED SAUER: Let’s see, we have another question here What’s the best strategy to respond to requests that probably take more than 60 to process? MANDY WAITE: Was that part of the question? So make a backend process to process the request That’s the best thing to do You can do these things out of band, or almost natural request itself So the user makes a request You need to actually service that request within 60 seconds, otherwise you’ll hit a deadline exceeded exception in both Java and Python runtimes That’s the request timeout we have that’s always in place for what we call frontend instances currently There’s no such limitation in backend instances So what Fred said about moving to the server model, if you choose to use a standard instance, so it’s not using autoscale, those things won’t apply It won’t automatically hit its request timeout So the best thing to do is if you need to do heavy lifting, then when working to answer a user request you should find a way to do that asynchronously out of band from actually responding to the user So long running task, let’s say user makes a request that will result in generating 100 e-mails to different people What you need to do is return a request back, respond to the user pretty quickly; and offline doing it in the background– the e-mail process using a backend instance It doesn’t need to be a backend instance, you could push it off the task queues, and then you can start using things like Compute Engine and suchlike Compute Engine has the ability to do pull requests using using task queues So you can use Compute Engine instances to service those

requests as well But yeah, basically take it away from the frontend instances and push it into a background somewhere, using the backend end process or Compute Engine FRED SAUER: Yeah, task queue is definitely the way to go The pull queues you mentioned are great for working with external systems, like doing off App Engine work, like Compute Engine, but you could really do it on a server that you have sitting in a rack in your office if you have some special processing that you need to do And then push queues are a great way of enqueing work asynchronously And those requests come back to your application as if it were traffic, but its internal traffic So App Engine will scale up and down if you create more requests And then you have a 10-minute deadline instead of the 60-second So you can easily make those long running URL fetch calls or do more complex processing MANDY WAITE: OK FRED SAUER: All right, when a user uploads a photo, like a movie or an audio clip, how do you store it so it can be statically available to your application? So it looks like they’d rather not have a dynamic request handler, where they have to handle the request and figure out what [INAUDIBLE] to serve, what file to serve But how do you make it available statically, so the user can always get to it without involving App Engine runtime? MANDY WAITE: So correct me if I’m wrong, but I guess you would probably find a place where you would actually store this photographs, other images and movies- that kind of blob data And then you would make sure that you declare that folder, that directory to be a static directory within the declaration of the application And that way all of that will be pushed to the edge by our edge servers FRED SAUER: Yeah, you could do that for pre-defined content If you’re doing dynamically uploaded content though, you’d probably do that through your Google Cloud Storage MANDY WAITE: Yes, basically storing in Google Cloud Storage and then you can provide a URL back to the user that can actually access it directly from Cloud Storage, as opposed to going through your App Engine instances FRED SAUER: Exactly, yeah MANDY WAITE: Yeah, good point FRED SAUER: And, yeah, we have a new Google Cloud Storage Client Library that you can find in the groups And it’s basically a new library for accessing Google Cloud Storage very easily and efficiently from App Engine So I’d probably use that to get the files into the Cloud Storage And then just like you said, [INAUDIBLE] the URLs directly to that MANDY WAITE: And it has the additional benefit of not actually hits in your instances So you won’t actually incur any costs, because you’ll be serving that data directly from Google Cloud Storage FRED SAUER: Yeah, you’re just paying for bandwidth, not CPU Yeah If I built my start up on App Engine, will Google help me with publicity? MANDY WAITE: Well, I guess, it depends how successful you are FRED SAUER: Depends on how cool an idea you have You’ve probably seen a number of customers with interesting stories on the App Engine blog I think if you came to us with a really cool story, just reach out We love to tell these stories and share them with our users So that’s probably– yeah MANDY WAITE: Tell us about your application Tell us about all the of the cool and interesting things you’re doing and we can look into it for you FRED SAUER: And also you and I both do presentations We often share partner stories and things like that So, definitely, tell us about them We’d love to feature them MANDY WAITE: Yeah, definitely FRED SAUER: Well, that was the last question of our list Looks like we’ve got to pretty much everything There is a couple that we skipped over due to time I want to thank all the viewers for visiting today Send us more questions if you really like this Let us know, because we can definitely do it again I think it was fun answering questions Thank you, Mandy, for doing the presentation at the beginning It was very helpful MANDY WAITE: Thank you, Fred And don’t forget Google I/O next week Follow it as live if you possibly can, if the sessions you are interested in are actually being streamed live If not, follow them on YouTube afterwards or the Google I/O sites The links, as I say, are in the the YouTube video of this recording And also, if you can, participate in Google I/O Extended So that’s really one way of getting involved in Google I/O outside of event itself FRED SAUER: Yeah All right Thank you MANDY WAITE: Thank you so much Cheers Bye-bye [MUSIC PLAYING]

You Want To Have Your Favorite Car?

We have a big list of modern & classic cars in both used and new categories.