My patch for XMLVM described previously has been commited to the head by the wonderous XMLVM team. The patch allows you to develop OpenGL application in Java that can be compiled into iPhone ObjC and then onto the real device.

Thats right, OpenGL development in a nice language like Java with a target of iPhone. Performance is pretty damn great too - especially when you manage the garbage carefully with the XMLVM annotations.

Thanks XMLVM team, great stuff!

I've been playing about with this XMLVM for a couple of weeks now. I just love the idea of using Java to produce iPhone stuff. More than that it adds nicely the suite of stuff I want to write that lets me have one code base and drive different platforms. One code base, one set of maintenance, one set of debugging and one set of bugs, should equal more time free to polish the heck out of games.

Anyway.. being that I've done quite of bit of OpenGL before I know that it's the best way to write polished feeling games. On the iPhone it appears to also be the only way to get reasonable performance bar drawing 50 or so small images to the screen. If you want to start upping that and blending, rotating etc you need OpenGL ES. What's more if you want to go to 3D games on Android/iPhone (though I probably don't) you gonna want GLES access. So, enough of a ramble about why, what has been achieved.

First the Java version of the iPhone compatibility library from XMLVM needed to have GL functionality exposed. This was pretty simple, a static class that exposes each of the methods and constants required right now. Next a new view type that can perform the setup required for GLES on the iPhone. I could have exposed the setup methods too but they're awfully murky and should be basically the same between all applications. So now, I have methods to code against. Port some iPhone GLES code (NeHe lesson 4) and ready to roll right?

public class NeHeLesson4View extends GLView {
	...
	
	public NeHeLesson4View(CGRect rect) {
		super(rect);
	
		viewWidth = rect.size.width;
		viewHeight = rect.size.height;
		
                setOpaque(true);
                setClearsContextBeforeDrawing(false);
	}

	private void initViewGL() {
	        ...
		
		GL.glMatrixMode(GL.GL_PROJECTION); 
		size = zNear * (float) Math.tan(
                        (TO_RADIANS * fieldOfView) / 2.0); 
		GL.glLoadIdentity();
		GL.glFrustumf(-size, size, 
                   -size / (viewWidth / viewHeight),
                    size / (viewWidth / viewHeight), 
                    zNear, zFar); 
		
		GL.glViewport(0, 0, (int) viewWidth, 
                             (int) viewHeight); 
		GL.glMatrixMode(GL.GL_MODELVIEW); 
		GL.glLoadIdentity(); 
		GL.glClearColor(0.0f, 0.0f, 0.0f, 1.0f); 
	}
	
	@Override
	public void renderView() {
		if (!viewSetup) {
			viewSetup = true;
			initViewGL();
		}
		
		GL.glClear(GL.GL_COLOR_BUFFER_BIT);

		GL.glLoadIdentity(); 
		// Triangle
		GL.glTranslatef(-2.0f,1.0f,-6.0f);
		GL.glRotatef(rtri,0.0f,1.0f,0.0f);	
		GL.glEnableClientState(GL.GL_VERTEX_ARRAY);
		GL.glEnableClientState(GL.GL_COLOR_ARRAY);
		GL.glColorPointer (4, GL.GL_FLOAT, 0, 
                                   triVertexColors);
		GL.glVertexPointer(3, GL.GL_FLOAT, 0, 
                                   triVertices); 
		GL.glDrawArrays(GL.GL_TRIANGLE_STRIP, 0, 3);
		GL.glDisableClientState (GL.GL_COLOR_ARRAY);
		
		// Square
		GL.glLoadIdentity();
		GL.glColor4f(0.0f, 0.0f, 1.0f, 1.0f);
		GL.glTranslatef(2.0f,1.0f,-6.0f);
		GL.glRotatef(rquad,1.0f,0.0f,0.0f);
		GL.glVertexPointer(3, GL.GL_FLOAT, 0, 
                                   quadVertices);
		GL.glDrawArrays(GL.GL_TRIANGLE_FAN, 0, 4); 
		GL.glDisableClientState(GL.GL_VERTEX_ARRAY);
		
		rtri += 1.5f;
		rquad -= 2.5f;
	}
}

Well, no, the simulator didn't yet support GL on the desktop. Integrating JOGL into the emulator took a little bit of hacking about, turns out JOGL 2 doesn't support anything less than OpenGL 2.0 - bit daft in my opinion. Hold On! Why is Kev using JOGL, he loves LWJGL. I do, but this seemed like an opportunity to refresh my JOGL knowledge and it was as painful as ever :) JOGL integrated, a bit of debugging later and the OpenGL compatibility layer is running in the Simulator:

Great, I can confirm the GL code is correct. Awesome. Next to get it working on the iPhone. First the objc version of the compatibility library needs extending again - adding implementation of java.nio.FloatBuffer etc and some more Core Graphics classes. Next over into OSX, cross compile, compile the objc and try the sucker. Hmmm.. not working, at all!

Turn's out I've made some dumb assumptions in the objc code for the GLView. Some few hours later I have it running in the emulator - just need to get it tested on a real phone now:

So again, XMLVM has allowed me to build stuff for the iPhone in Java. It's proven to be not too tricky to update to add functionality as and when you need it. I've submitted a patch to add this stuff (including the example) into project. Hopefully we can work through the integration and get it out there for everyone.

So far I can only encourage people to grab XMLVM and get playing!

Spent a few evenings working on some extra features for the abstraction that I need for the football game. The additions of course have to work across all three platforms so it took some time. However, there is now:

• Sound FX - basic load sound and play back. (AudioClip/SoundPool/AVAudioPlayer)
• Persistant Storage for game settings, save games, etc - (Cookies/Preferences/NSUserDefaults)
• Suspend/Resume functionaltiy for phone based games (No Applet Support/Bundle/NSUserDefaults+NSData)

It all seems to work but I ended up spending a good evening and half chasing round bugs on the IPhone's lifecycle and data storage.

Still... back to the game now!

Last couple of days I've spent a lot of time fiddling with game play, adding new rules and tuning the AI. It's all starting to come together. As a bit of a relief tonight I added a bunch more international teams and pixeled them all up (strips and flags). This is the first game where the pixel art is all from me (albeit aided by follow the templates for proportions and colours provided elsewhere).

Here's the new team selection:

The portable game code stuff is really interesting, I'm still playing and working on it at the moment. A few people have picked up the stuff I pushed out a few days ago and have started experimenting and porting some of their stuff. We've all found the same thing for the most part, it's possible but there's plenty missing from XMLVM which requires updates to both the objc and the xsl.

Hopefully the more of us that are using it, the more we'll understand and the more will get done. With that in mind here's a new zip which contains an updated version of the objc and xsl (recent changes) and a new structure for the XMLVM project which contains only the iphone portions and allows rebuilding of the XMLVM jar from a nice simple Ant script. In addition I've written a simple Lint tool that scans the outputed objc for common errors which gives a little more chance of the code compiling later.

So what else. Well Adam was nice enough to try this stuff out and has started porting some of his stuff over. He's sent me some images of the stuff he's working on:

And a video:

Quick Video of IPhone App in Java from Kevin Glass on Vimeo.

I've spent most of the weekend getting my Java RPG (which works on applet/android already) functioning correctly on IPhone. This was mostly to stretch the objc implementation a bit since the RPG is a considerably bigger and more complicated piece of code. In includes XML parsing and reading arbitrary resources along with some procedural generation stuff. The XML parsing was the most fiddly bit. The translation process for my the normal DOM was too painful, so I tried MXP1. Unfortunately they do some clever stuff inside that I couldn't replicate in objc with my limited skills. So, I ended up writing a naive XML parser that just relies on InputStream - works a treat.

So, now my RPG is running on iPhone, performance is pretty good so far and the objc library has grown a lot because of it. Anyway, enough words, here's some more screenshot:

It's still really fun, but I think I'm back to focusing on the footie game for a while now. Also, it might be time to get some proper sleep.

Quick update on the portable game code stuff. I'd already seen it running on a real applet (well duh) and a real android handset (kinda neat - thanks Dave!) but not on a real iphone. Thanks to John in Oz for this wonderful photo:

Showing the test stuff I released last week built and running on a real iphone. There's a little worry on the legality of releasing this stuff based on the iPhone Developer Agreement but I've yet to understand what the issue is.

Still, great news! Real hardware!

Update: Legality issue was a misunderstand. The objc code is still plain objc so it's just a plain app. Yay for XMLVM again!

I've had a bit of feedback so far, most people just want to try it for themselfs. So, I've hacked up a distribution and a little test "game" (since I don't want to give my soccer game source away). I haven't had a chance to build this one for android and iphone, but it's considerbly more primitive than soccer so should work just fine. The test game is available as an applet already.

From the distribution readme:

Applet/Android/IPhone Portable Code
-----------------------------------

This is just a quick snapshot of where I am with this stuff. Build instructions below 
and please remember this isn't really for use unless you already know a fair bit
about what you're doing. There is no support.

Note that the majority of this work is based on the http://www.xmlvm.org project.

There may be troubles with project configuration. If you're not experienced 
enough to resolve class path errors in projects you're likely to get stuck later on. 
Stop now :)

Setup
-----

There are 6 eclipse projects included.

aboid - The Game API
aboid-applet - An implementation for applets in Java2D
aboid-android - An implementation for android using the Canvas API
aboid-iphone - An implementation for iphone making use of XMLVM
objc-compat - An update version of the XMLVM compatibility library 
             (note that this is actually included in xmlvm.jar)
TestGame - a simple test game to show the system

Import these projects, they have interdependencies. Note that the aboid-iphone project 
must export the objc-compat project and that the aboid-android project must export it's 
android.jar library so that the game can rely on their classes.

Building Applet
---------------

An included ant script build's a versioned applet JAR. Note that the ant script must be 
configured with the location of your workspace (and hence all the project) and the 
location of the xmlvm.jar.

The "build" target creates the applet JAR

Building Android
----------------

I use the Android Eclipse integration to build the Android project and test on the 
emulator. Note that you must generate the R.class file locally to the TestGameActivity 
class included in the TestGame project.

Building IPhone
---------------

An included ant script build's the ObjC. Note that the ant script must be configured 
with the location of your workspace (and hence all the project) and the location of 
the xmlvm.jar.

The "iphone" target generates the ObjC code to the "target/iphone/output" directory. 
On OSX run "make" in the directory containing the generated ObjC to build the IPhone 
application (assuming you have the IPhone SDK installed). The make file also tries 
to run the emulator. You need to run the produced object from the command line to 
get stdout, rather than through the emulator.

The distribution is available and based on source from previous articles. Note that it includes an extended version of the XMLVM project.

Let me know how you get on, but please don't expect support. It's way to early for bugs. I'll be trying to build the test game against IPhone and Android in the next couple of days, let me know if you beat me to it :)

UPDATE

Ok, confirmed the test app builds for applet, android and iphone. If you want to use RuntimeException just copy the java_lang_Exception.h/m and rename. Confirmed running on all 3:

Earlier I posted some details of what I've been working on for a few days, a portable system that allows me to code once and deploy to applet, android and more complicatedly - iPhone. The first two are reasonably easy to get to, they're both Java and both support the majority of the SDK. They have different rendering methods (if you want to use their 2D APIs) so that needed to be abstracted, but that was basically it.

However, when it came to the iPhone the code needs to be in ObjC (or C++/C), so the Java will have to be converted - for me it has to be automatic. What's more somehow the Java code needs to have access to the iPhone SDK methods. Sounds like quite a tall order? Step in XMLVM, it's a tool that provides for both of the tasks above (along with a great deal of other stuff).

First, XMLVM gives you Java to ObjC conversion. More explicitly it converts from Java byte code to an intermediate XML form and then passes that through an XSL sheet to produce whatever language the sheet is designed for. In this case there's an XSL to produce ObjC. However, the current version of the sheet is just a start on the task - it's missing a bunch of byte codes. As I've gone along with the process I've been adding one's I've needed, only four or five. A typical mapping looks like this:

<xsl:template match="jvm:ishr">
  <xsl:text>    _op2.i = _stack[--_sp].i;
    _op1.i = _stack[--_sp].i;
    _stack[_sp++].i = _op1.i >> _op2.i;
</xsl:template>

The conversion process has mostly already been provided by the great XMLVM authors. The bit that really takes the time is the ability to make use of iPhone SDKs in Java, this has a long way to go. XMLVM looks a bit like this in that regard:

So your game code sits up there in the Java space. To be able to build it against the the iPhone SDKs there has to be a Java version of those APIs implemented. Those are provided as part of the XMLVM compatibility library. Whats more is that these libraries arn't just empty API/prototypes for the methods you need, but rather are implemented in terms of Java 2D. This means that once you've built your game using these APIs you can actually run it on the Java side and test the basic logic before you even get to the ObjC problems. My game for instance looked like this after I'd built against the Java libraries:

It looks sort of like iPhone right? XMLVM provides some internal classes that emulate the iPhone. Be warned however, this isn't the real iPhone emulator by any means. It's just a Java shell that let's you test. The code will only be able to run on the real emulator once the ObjC has been produced and built.

The ObjC version of the compatibility library provides a the API functions to the converted Java code. So, once the Java game code has been translated as described above it can be built against the ObjC compatibility library to produce the final iPhone application. The nice thing here is that once you're at that stage there's no difference between your ObjC application and any other hand coded one - which means no legal qualms, no jail breaking, no problems.

There's a problem of course, this ObjC library has to provide access to every iPhone SDK method/function you want and so far there arn't many implemented. For us Java types implementing the Java version of each API function is reasonably easy, however then you get the joy of dropping into ObjC and writing the equivalent function there. It's probably clearer with an example. Take the getClip() method on the CGContext API, in Java it looks like this:

public CGRect getClip() {
    Rectangle clip = graphicsContext.getClipBounds();
    
    return new CGRect(clip.x, clip.y, clip.width, clip.height);
}

Easy right? Just take the Graphics2D clipping bounds and convert it to a suitable type. However, the ObjC version is just a bit more dirty:

- (org_xmlvm_iphone_CGRect*)getClip;
{
	CGRect rect = CGContextGetClipBoundingBox(context);
	org_xmlvm_iphone_CGRect* retVal = 
          [[[org_xmlvm_iphone_CGRect alloc] init] autorelease];
	retVal->origin->x = rect.origin.x;
	retVal->origin->y = rect.origin.y;
	retVal->size->width = rect.size.width;
	retVal->size->height = rect.size.height;
	
	return retVal;
}

It's not that bad, a bit ugly and a bit cumbersome. The problem is finding these methods, while the documentation isn't bad it's having a clue where to start. This is probably my lack of iPhone experience though - I don't have any at all. Essentially all it does it map the method to a CGContext C API call and return the result.

Once the ObjC API is implemented it's just a matter of take the generated ObjC and the ObjC library, build them on OSX using the iPhone SDK and out comes your application. Of course there will be problems, and debugging applications on the iPhone isn't much fun. I've mostly relied on running the application from the command line and getting stdout. Here's my game running on the real iPhone Emulator:

There are some limitations that I've come across, I guess some will be overcome with time and patience, but they're probably worth mentioning:

• System.arraycopy isn't implemented and I couldn't see an easy way to do it. Shouldn't hurt most people but I use it now and again
• No field names can be duplicated in child/parent class. All members end up public so can't compile with the same name
• Array references are never null - so you can't use a null check on int[] for instance

So that's XMLVM. It's big and it's clever and it gives us a way onto the the iPhone platform. If I could get hold of the developer's I'd contribute the stuff I've added back (it's available below). If we could only get this set up as a community based project I'm sure we'd soon have most of the byte codes mapped and the majority of the Java API and iPhone SDK made available in the compatibility library!

For now it's hopefully enough to say thank you to the XMLVM developers. It's a really great piece of technology and I hope it carries on soon.

Updates to XMLVM including a new build

I had a few days off, went to the zoo with Meg, shopping with Cath, castles and general time out and about in Sun. Nice! Every spare minute and late into the night I was of course coding away. I've been looking at simple rendering API to abstract between Applets and Android, this was mostly because I find the Android tooling a bit heavy to work with. Being able to test and debug against an applet version and then rebuild and run simple tests on the Android emulator/handset would be really good for productivity.

Since Android and Applets both run Java it wasn't that difficult. Built an extremely simple game API which lets me just get input and splat images to the screen (ok and a bit of geometry and state management). Implemented this in terms of Java2D (for applets) and the Canvas API (for Android). Excellent, I can now code the game once and build as an applet or an Android acitivty. There are of course caveats:

• Android only allows a 16mb heap per application. Gotta be careful how big those structures you build are
• Android performance isn't desktop, so again be careful. For my turn based stuff that doesn't hurt too much
• Android GC is much slower than normal Java, of course it's just good practice not to create to much garbage

So it's the same set of Java code runs in both Android and Applet. With the applet you need to specify a game class in the parameters like this:

<applet archive="soccer-8.jar" 
           code="org.newdawn.game.applet.GameApplet" 
           width="320" height="480">
	<param name="gameClass" value="org.newdawn.soccer.SoccerGame"/>
</applet>

In Android there's a tiny bootstrap Activity class that looks like this:

public class SoccerActivity extends Activity {
	private GameView gameView;
	
    /** Called when the activity is first created. */
    @Override
    public void onCreate(Bundle savedInstanceState) {
        super.onCreate(savedInstanceState);
        
        requestWindowFeature(Window.FEATURE_NO_TITLE); 
        
        getWindow().setFlags(WindowManager.LayoutParams.FLAG_FULLSCREEN,
                             WindowManager.LayoutParams.FLAG_FULLSCREEN);          
        setContentView(R.layout.main);

        gameView = (GameView) findViewById(R.id.gameview);
        gameView.setAssets(getAssets());
        gameView.setGame(new SoccerGame());
        gameView.start();
    }
}

There's still a bit of work left to optimise the game sufficiently for Android, the AI takes a while to run. However, I started thinking about marketing this game. The IPhone is the hottest market for mobile games at the moment. I believe the Android market will take over, but right now it'd be ideal if I could publish to IPhone as well. However, key to me again is:

• I like Java, I find it very productive. I don't particularly want to alot of work in Obj-C
• One code base is better for me, one set of bugs to fix
• Being able to test the same code an applet, before having to go to my Mac and IPhone is better

Enter XMLVM, it's a tool that converts from Java byte code into an XML document, and then translates this into other languages - one of these being Obj-C. Ok, it's not really as simple as that but I'll describe the details in the next post. Suffice it to say what you get from XMLVM SVN while brilliant doesn't contain enough to write even simple games like Funky Football.

XMLVM relies on a compatibility library written in both Java and Obj-C to provide access to the IPhone platform APIs. The Java one is to build against (and to debug against). The Obj-C version is whats built against once the rest of the code has been converted automatically. So, first I extended the Java side to support the features I needed, this let me use the XMLVM Java version of the IPhone emulator:

Brilliant! At least I thought it was until I realised I had to then implement the features I'd added to the Java library in Obj-C. Scary when you don't know anything about the language really. So I boot my Mac, start trying to use XCode. That really is a poor development tool, especially when you've used Eclipse and Netbeans. It took me a day or two to realise that the nice people at XMLVM had made the output compilable from the command line. This also gave me access to stdout for debugging (more on that in the next post). So I've now spent two days (spare time coding of course) hacking up the bits and pieces of the API I need to make available to Java. This includes SDK things that arn't implemented like ArrayList.indexOf(). This isn't actually as bad as it sounds once you get going. The documentation for the IPhone platform is pretty reasonable and there is a pattern to it. I can now honestly say I know more about Obj-C than I ever wanted to.

It's mostly working now, theres a bit of garbage collection issue with objects that you never assign to a variable (they get collected too early) but otherwise it's all looking pretty good. The shot shown in the rather gratuitous overview below is actually taken from the OSX IPhone emulator running the produced code. You can actually play the game too :)

The IPhone bootstrap class is light, just creates a UIApplicationDelegate for the game:

public class IPhoneSoccer extends IPhoneMain {
	public void applicationDidFinishLaunching(UIApplication application) {   
		setGame(new SoccerGame());
		
		super.applicationDidFinishLaunching(application);
	}
	
	public static void main(String[] argv) {
		try {
			UIApplication.main(argv, IPhoneSoccer.class);
		} catch (Throwable e) {
			System.out.println(e.toString());
			System.out.println(e.getMessage());
		}
	}
}

So what have I achieved? Well, i've taken a reasonably simple game and API and successfully made it possible for one Java project to compile to all 3 platforms (Applet/Android/IPhone). They all work (to some extent) and there's nothing license infringing or jailbreak requiring about any of it.

What next? I've some bugs to sort out (the font issue above is already resolved for insetance), some more SDK to implement and a lot of code to contribute back to XMLVM (should I ever be able to get hold of the developers!). Then I need to try out my bigger more complicated game (Runes of Yore, an RPG) through the system. There are bound to be extra things to find. I'm also going to do another post covering more about how XMLVM works in this use case, and what needed to be extended.

Hopefully someone will be interested in the post and give some time to do this whole thing properly! It all looks so good, but there's about 3 months full time work here to make a working version that supports the majority of the SDK.

The renderer/input framework for both Yore and Funky Football is a very simple abstraction layer. It allows some basic things (draw images, rectangles, get mouse input) but it's enough to be getting little games going with. This has made it easy for me to get some applets going over the last few weeks.

Tonight I spent some time porting that little layer over to android. It's been a long night but I now have one project which builds both android and applet artefacts - including run configurations in eclipse. There's a tiny Activity bootstrap for the android version but all the core code is the same.

There are more issues to face due to memory constraints on android (16 meg heap) and the amount of memory my AI search tree is taking up (some 20k nodes in some cases). Nice to see it playing on the emulator at least though :)

The AI still needs more work and I've had a bunch of useful feedback, so I can see this game has at least a few more days of legs on it. All in all it's been an effective way of getting the android port done without things getting too complicated too quickly!