Google App Engine – The Backend (1)

Hey everyone!

So here we go with Part I of the series! Again, this post will focus on defining a very simple object (in my case, I’ll use a very simple VideoGame object which I used for my actual application) and then creating a series of wrappers in your Google App Engine project to allow you to add/remove/update entries in your server’s database. But before I get ahead of myself – let’s start from the beginning.

So what exactly is Google App Engine and why do I need it?

From their website:

Google App Engine enables you to build and host web apps on the same systems that power Google applications. App Engine offers fast development and deployment; simple administration, with no need to worry about hardware, patches or backups; and effortless scalability.

Simply put, setting up a Google App Engine project allows you to create a server that is hosted in the “cloud”. This server comes with a very convenient and scalable database, and gives you the ability to interact with this database through standard HTTP requests.

Why is this useful as an Android developer? Well, aside from the obvious answers of the necessity for having a centralized database in order to develop any kind of large scale data-centric application, because any Android application is limited by the memory and processing powers of the phone itself, it’s far from ideal to do any kind of heavy data scraping or to maintain any kind of large SQLite database on the phone.

Now time to dig into the code.

Let’s start with my simple object class called VideoGame – note that this object class will look slightly different from the normal Java class object as we need to define a couple of annotations which allow the object to be stored in Google App Engine’s database:

import javax.jdo.annotations.PersistenceCapable;
import javax.jdo.annotations.Persistent;

import com.google.appengine.api.datastore.Link;

@PersistenceCapable
public class VideoGame extends ModelBase {

    // note the Persistent tags
    @Persistent
    private String name;

    @Persistent
    private Link imgUrl;

    @Persistent
    private int consoleType;

    public VideoGame(String name, String url, String consoleType) {
        this.name = name;
        // note the App Engine class Link - simply a wrapper around URL strings
        this.imgUrl = new Link(url);
        this.consoleType = VideoGameConsole.convertStringToInt(consoleType);
    }

    public String getName() {
        return name;
    }

    public void setName(String name) {
        this.name = name;
    }

    public Link getImgUrl() {
        return imgUrl;
    }

    public void setImgUrl(Link imgUrl) {
        this.imgUrl = imgUrl;
    }

    public int getConsoleType() {
        return consoleType;
    }

    public void setConsoleType(int consoleType) {
        this.consoleType = consoleType;
    }

    public static class VideoGameConsole {

        public static final String XBOX = "Xbox";

        public static final String PS3 = "Ps3";

        public static final String WII = "Wii";
        
        public static final String[] CATEGORIES = { XBOX, PS3, WII };
        
        public static int convertStringToInt(String type) {
            if (type == null) { return -1; }
            if (type.equalsIgnoreCase(XBOX)) {
                return 0;
            } else if (type.equalsIgnoreCase(PS3)) {
                return 1;
            } else if (type.equalsIgnoreCase(PS2)) {
                return 2;
            } else {
                return -1;
            }
        }

        public static String convertIntToString(int type) {
            switch (type) {
                case 0:
                    return XBOX;
                case 1:
                    return PS3;
                case 2:
                    return PS2;
                default:
                    return null;
            }
        }
    }

}

And just quickly here’s the ModelBase class that it extends:

import javax.jdo.annotations.IdGeneratorStrategy;
import javax.jdo.annotations.IdentityType;
import javax.jdo.annotations.Inheritance;
import javax.jdo.annotations.InheritanceStrategy;
import javax.jdo.annotations.PersistenceCapable;
import javax.jdo.annotations.Persistent;
import javax.jdo.annotations.PrimaryKey;

@PersistenceCapable(identityType = IdentityType.APPLICATION, detachable="true")
@Inheritance(strategy=InheritanceStrategy.SUBCLASS_TABLE)
public class ModelBase {

    // in any standard database, each row needs to have a unique ID
    // this ModelBase class defines a long ID that each VideoGame object will inherently have
    @PrimaryKey
    @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
    private Long id;
	
	public Long getId() {
		return id;
	}
}

Now, it might seem like a lot of code, but let’s just break the important parts down piece by piece. First off in the ModelBase class we’re simply defining a generic class of objects which contain an ID. Like in all databases, each row must have a unique ID so that it can be identified, and in our case since Google App Engine uses a JDO (Java Data Objects) database (meaning rows are represented as Java objects) each new object we create must have its own ID. If you have any background in SQL then you’ll immediately recognize key words like PrimaryKey which will give you a big hint at what this initial ModelBase class is doing. You’ll also see that there’s a @Persistent tag, and in Google App Engine, when you define a field to be persistent you are simply telling the database to store those values. In other words, you’re essentially defining a new column in your database.

Now, once you understand these initial basics, the rest is pretty self explanatory. Here in my VideoGame class, I simply extend ModelBase to make it a valid row object with a unique ID (this ID will automatically get set for you upon adding it to the database) and then declare the object @PersistenceCapable. Again, this is simply telling the Google App Engine database that this VideoGame object should be store-able, or going back to SQL speak you’re basically creating a new table called VideoGame here which will store VideoGame rows.

I then make a couple of fields persistent (i.e. defining columns in my table) and add some simple helper methods and voila! Now I can go ahead and start adding these VideoGame objects into my database (for a less verbose and more technical set of explanations, Google has posted this tutorial, also very good).

The code for the JDO wrapper (as I call it) is:

// wrapper for storing/deleting/updating VideoGame objects
public class VideoGameJDOWrapper {

    // insert a single video game
    public static void insertGame(VideoGame g) {
        // check to make sure this game doesn't already exist
        boolean exists = checkGameExists(g);
        if (!exists) {
            PersistenceManager pm = PMF.get().getPersistenceManager();
            try {
                pm.makePersistent(g);
            } finally {
                pm.close();
            }
        } else {
            System.err.println("ERROR: Duplicate attempt");
        }
    }


    // when adding multiple video games, a batch insert is more efficient
    public static void batchInsertGames(List<VideoGame> games) {
        List<VideoGame> toInsert = new ArrayList<VideoGame>();
        for (VideoGame g : games) {
            boolean exists = checkGameExists(g);
            if (!exists) {
                toInsert.add(g);
            }
        }
        PersistenceManager pm = PMF.get().getPersistenceManager();
        try {
            // only add the games which don't exist already
            pm.makePersistentAll(toInsert);
        } finally {
            pm.close();
        }
    }


    // example for retrieving rows - in this case by platform
    public static List<VideoGame> getGamesByPlatform(String platform) {
        // find the numeric type of the platform
        int type = VideoGameConsole.convertStringToInt(platform);

        // open the persistence manager
        PersistenceManager pm = PMF.get().getPersistenceManager();

        // initiate a new query for the VideoGame class
        Query query = pm.newQuery(VideoGame.class);

        // in SQL - this filter is equivalent to a WHERE statement
        // essentially, here we have a column called consoleType (defined earlier)
        // and we are about to pass it a value
        query.setFilter("consoleType == inputType");

        // define what parameter you're going to input
        query.declareParameters("int inputType");

        List<VideoGame> ret = null;
        try {
            ret = (List<VideoGame>) query.execute(type);
        } finally {
            query.closeAll();
        }

        // at this point, you should have a List of VideoGame objects
        return ret;
    }


    // deleting all games with a certain console type
    public static void deleteGamesByType(String platform) {
        int type = VideoGameConsole.convertStringToInt(platform);
        PersistenceManager pm = PMF.get().getPersistenceManager();
        Query query = pm.newQuery(VideoGame.class);
        query.setFilter("consoleType == inputType");
        query.declareParameters("int inputType");

        // up to this point, defining the query is identical to the GET request above
        // only difference is in the method we call with this query
        query.deletePersistentAll(type);
    }


    // wrapper to check if a video game exists
    private static boolean checkGameExists(VideoGame g) {
        PersistenceManager pm = PMF.get().getPersistenceManager();

        // I defined an identical game if it had the same name and was on the same console
        Query query = pm.newQuery(VideoGame.class, "name == gameName && consoleType == gameType");
        query.declareParameters("String gameName, int gameType");
        List<VideoGame> ret = null;
        try {
            ret = (List<VideoGame>) query.execute(g.getName(), g.getConsoleType());
        } finally {
            query.closeAll();
        }
        return !ret.isEmpty();
    }

    
    // if you have the row ID of the VideoGame then
    // there's a simple way to retrieve the associated object without defining a query
    public static VideoGame getVideoGamesById(long id) {
        PersistenceManager pm = PMF.get().getPersistenceManager();
        return (VideoGame) pm.getObjectById(VideoGame.class, id);
    }


    // this is also the most practical way to update an object
    public static boolean updateVideoGameName(long id, String newName) {
        PersistenceManager pm = PMF.get().getPersistenceManager();
        boolean success = false;
        try {
            // how the persistence manager works is that as long as it is OPEN
            // then any chances to an object will AUTOMATICALLY GET UPDATED AND STORED
            VideoGame v = (VideoGame) pm.getObjectById(VideoGame.class, id);
            if (v != null) {
                v.setName(newName);
                success = true;
            }
        } catch (JDOObjectNotFoundException e) {
            e.printStackTrace();
            success = false;
        } finally {
            // however once you CLOSE the persistence manager, then any updates to the object
            // will not be reflected in the database
            pm.close();
        }
        return success;
    }

}

And again for the sake of completeness here’s the code for the Persistence Manager (taken straight from the Google App Engine tutorial):

import javax.jdo.JDOHelper;
import javax.jdo.PersistenceManagerFactory;

public final class PMF {

    // this is a singleton which acts as a gateway to the database itself
    private static final PersistenceManagerFactory pmfInstance = JDOHelper
            .getPersistenceManagerFactory("transactions-optional");

    private PMF() {
    }

    public static PersistenceManagerFactory get() {
        return pmfInstance;
    }
}

So there you have it! Everything hopefully should be pretty self explanatory. But just to summarize, I’m going to assume that most people out there have some experience with SQL (or variants of SQL) and so I’ll pair each step in the Google App Engine process with its equivalent step in setting up a SQL database:

1. Defining a Persistence Manager -> Opening a connection to your SQL database

2. Defining a PersistenceCapable object -> CREATE TABLE in SQL (and like in SQL – you need a column for Primary Key)

3. Defining Persistent fields -> creating columns in a SQL table

4. Defining a Query object that returns/deletes a list of Java objects -> defining a SQL query that returns/deletes a sub-table of rows

5. Specifying a Query object’s filters -> including a WHERE filter in a SQL query

6. Retrieving an object from the Persistence Manager and changing a field before closing the PM -> retrieving a row in SQL and updating a field

And I think that just about wraps it up – probably my longest post to date and yet I feel like I could still go into more detail for a few parts, but hopefully you guys get the picture.

Happy Coding! Part II up next.

– jwei