Monday, April 21, 2014

Pyroscaphe: Adding a Touch Screen to PC Gaming

I have a confession to make.  I am a big fan of PC gaming.  Ok, that's not too controversial, but let me take this a step further.  I am a big fan of PC gaming...with a controller.

For all too long, the controller has been a second-class citizen in PC-land, with the mouse and keyboard reigning supreme.  This is why I was so excited when I heard news of the Steam controller that Valve is developing.  For those of you that don't know, this is a controller, currently under development, which is designed from the ground-up to work well with PC games, even those that traditionally could only be played with a keyboard and mouse.

Originally announced Steam controller design
The initial prototypes of the controller were impressive.  Obviously the dual trackpads are integral, but the part that excited me the most was the touch screen display at the center of the controller.  This touch screen was to be a multi-purpose input/output mechanism for games to utilize for auxiliary tasks or just as "soft buttons".

There are so many possibilities for a mechanism like this.
  • on-screen maps
  • inventory management
  • quick save/load buttons
  • ammo status for "realistic" shooter games
  • any auxiliary function that may not map well to existing controller inputs
To my disappointment, a few months ago Valve announced that the touch screen had been dropped from the steam controller design.  I'm sure they had good reasons to do this, but as an engineer excited by this concept, I just couldn't let this die.

Steam controller with touch screen removed

The Premise


Most of us already have powerful touchscreen devices in our pockets.  Rather than design a new controller with a built-in touch-screen, we could leverage the devices that we already have to enable these new types of game interactions.

That was my general line of thinking, and it's how Pyroscaphe was born.

Prototype Controller


Pyroscaphe


At its core, Pyroscaphe is a web server that allows you to interact with your gaming machine on a local network from a web-enabled device (like a phone or ipod touch).  This server serves web page UIs for a given game and also receives WebSocket requests which lets the player interact with the game in a real-time, low-latency manner.

In its current state, Pyroscaphe is mainly useful for sending keypress events to games, so through the UI on your mobile browser, you can essentially press keys on your keyboard.

In the future, I would like to make this more of a 2-way interaction, with the touch interface receiving more data about the running game.

Demo


(I apologize for the blair witch photography)



Why Web-Based?


There have been many articles recently that have declared the mobile web dead...or dieing, with the App being king.  However, for this particular application, I believe web-based UIs are best.

If something like this is to succeed, it depends heavily on the community generating content in an incremental, and distributed way.

This would be very challenging for a traditional mobile app.  With an app, we would have to have a central location for UIs to be reviewed and vetted, and they would have to go through the long Apple review process.  This process discourages frequent updates and also means that there must be a central authority that spends a lot of time hand-holding the process and vetting these touch UIs.

Since Pyroscaphe is essentially a locally-hosted web-app, it doesn't have these particular issues.  All of the content lives on the user's PC, where they have full control over what's installed and can update it incrementally with UIs created by the community.  There doesn't have to be any central authority.  UIs can be freely downloaded from ftp sites, discussion forums, or traded on floppy disk :)

Also, though inter-browser compatibility can be challenging at times, it's not too difficult to write HTML that works across most mobile browsers (as opposed to writing one interface in ObjectiveC, another in Java, etc..).

Finally, by using HTML, it greatly reduces the barrier to entry for creating one of these UIs.  HTML is a universal language that just about anyone can pick up fairly easily.

Client web page creation


Like I said previously, if Pyroscaphe is to have any chance to succeed, it must leverage the power of the community.  There are just so many talented and creative gamers out there.  If someone feels the itch to create a touch UI for their favorite game, I want it to be as easy as possible to do so.

One of the things I'm doing to accomplish this is to leverage HTML5 "data-" attributes.  Here is what my prototype "Duke Nukem 3D" page looks like:




<html>
  <head>
    <title>Duke Nukem 3D</title>
    <script src="/pyroscaphe.js"></script>
    <script> $(document).ready(function(){ initPyroscaphe(); });    </script>
    <style>
      <!-- CSS omitted for expository purposes -->
    </style>
  </head>
  <body>
    <table id="gunTbl">
      <tr>
        <td><img class="dukeGun" data-key="2" src="pistolIcon.png"></img></td>
        <td><img class="dukeGun" data-key="3" src="shotgunIcon.png"></img></td>
      </tr>
      <tr>
        <td><img class="dukeGun" data-key="4" src="chaingunIcon.png"></img></td>
        <td><img class="dukeGun" data-key="5" src="rpgIcon.png"></img></td>
      </tr>
    </table>
  </body>
</html>
As you can see, this HTML is extremely simple.  The bootstrapping mechanism is fairly lightweight as well.  Pages must pull in "pyroscaphe.js" and initialize Pyroscaphe when the document is loaded.

With that in place, pages can use the "data-key" attribute on an element to specify which key they want to press when a given element is touched.  In this case, weapons are selected with the number keys, so we bind the pistol image with key "2", shotgun with key "3", etc.

Behind the scenes Pyroscaphe uses jQuery to add the appropriate event handlers to send the right WebSocket commands to the server.

The prototype UI for Mortal Kombat is even simpler:



<html>
  <head>
    <title>Mortal Kombat</title>
    <script src="/pyroscaphe.js"></script>
    <script> $(document).ready(function(){ initPyroscaphe(); }); </script>
    <style>
      <!-- CSS omitted for expository purposes -->
    </style>
  </head>
  <body>
    <img id="finishHim" data-seq="V;A;L;V;E;" src="finishHim.png"></img>
  </body>
</html>
In this example, Pyroscaphe is bootstrapped just like before.  However, we just have a single image on the page with a "data-seq" attribute (as opposed to the "data-key" attribute that the previous page used).  As you've surely guessed, this tells Pyroscaphe to send a sequence of key presses to the game when this UI element is touched.

These are just 2 examples of very simple UIs that can be created.  Again, this is about leveraging the creativity of the gaming community to create these UIs.  The easier I can make it to create this content, the more likely it is that someone will do so.

Latency


Web apps have a bad rap for being slow and unresponsive, at least compared to native applications.  However, Pyroscaphe has several things going for it on this front.

It's designed to be used in a local network, so very few hops are required to get from client to server.  This also means interaction is more deterministic since we're not just sending data out into "the cloud", it's only going out on a small, controlled network.

We're also using WebSockets for all time-critical game interactions.  This means we don't have to deal with the overhead of creating a new socket connection for each server call (which itself involves a few round-trips).  We also don't have to deal with the overhead of the HTTP protocol.  As such Pyroscaphe has a very simple bare-bones command protocol that keeps performance snappy.

To validate that the latency wasn't too bad, I wrote a fairly simple test.


By round-trip time, I mean that this is the time it took for the UI to send a command to the server and for the server to send a simple response back.  So in reality, the lag between interacting with the touch UI and seeing a key press is roughly half of this time.  As you can see, in my tests the vast majority of calls had a round-trip time of 3-4 ms, which is more than sufficient for most game applications (and in-line with standard wireless controller latency).

A secondary benefit of using a single WebSocket (as opposed to many web-service calls with their own sockets) is that the data is serialized and we don't have to worry about data arriving at the server in a non-deterministic order.

FAQ


Won't the mobile web browser UI get in the way?


This can happen with some browsers.  I personally recommend using browsers that have a good fullscreen mode (like Atomic Web Browser Lite).

A touch screen on the controller diverts attention away from the action on-screen.  Won't that hurt the overall experience?


Touch screens are probably not the best input mechanism for really fast-paced action games.  However, there are a large number of games out there that don't require absolute non-blinking focus.  Those are the games that Pyroscaphe is intended for.

Can Pyroscaphe UIs send xbox 360 controller (XInput) button presses?


Not at the moment.  Currently, you have to use a program like Joy2Key to have your controller send key presses (and configure your game for keyboard input).

I would certainly like to get this working at some point.  However, I couldn't find an easy way to do this.  I could probably create a driver for this purpose (I personally work a lot with drivers by trade), but that's more trouble than I wanted to go through for the initial 0.1 version of Pyroscaphe.

Why are your demo UIs so ugly?


Sorry, UI design isn't my strength.  I'd be thrilled if someone wanted to help make these demos look better (or create new UIs).

So this can only send keypresses to a game?  So what.


It's not much right now, but I would like this to be the start of something better.  It's much easier to get keypresses to a game than to get data from a game (like showing an interactive map with your current location).  I suspect that doing the latter will require some game modding which isn't something I've done before.  Again, any help is welcomed.

Future Direction


There are a lot of possibilities for Pyroscaphe in the future.  Here are a few ideas I've had.

  • Auto-detect the game that has been launched and instruct the touch-screen to change UIs.
  • Simulate controller key presses via XInput and/or DirectInput
  • Simulate a mouse touchpad for navigating complex game menus
  • Detect key bindings or otherwise make it easier for users to map a given touch UI to their key bindings.

Conclusion


It's not much right now, but my hope is that if there are others like me out there, Pyroscaphe might gain some momentum and eventually lead to some really interesting and engaging gaming experiences.

Give It a Try!

Pyroscaphe's code is open source and freely available on GitHub.

https://github.com/justindvs/pyroscaphe

Don't want to build it from source?  You can download a binary distribution for Windows x64 here:

https://github.com/justindvs/pyroscaphe/releases

Wednesday, August 21, 2013

A Handful of Emacs Utilities

Every now and then, I like to take stock of what's in my .emacs file.  It reminds me of a bush in my yard.  I see it so frequently that I don't notice it growing steadily over time.  One day it just hits you that this thing is big and scraggly and needs to be trimmed way back.

I've learned the hard way that if wait too long before trimming back a bush, there's no going back to the way it used to be.  After trimming off several years worth of growth, you realize that it's not that nice little bush anymore.  It's a big bush that's been hacked into an ugly nub of its former self.

So, in an effort to avoid this situation (sometimes referred to as emacs bankruptsy), I periodically go through my .emacs file, pruning unused functions and reshaping the ones that need a little TLC.

I thought it might be interesting to share some of the utilities I've accumulated.  They have remained in my .emacs file over the years, winning the evolutionary battle to avoid being pruned.  Some are very small and self-contained, a few are slightly more ambitious.

Note: Some of these have been in my .emacs file for a long time.  There may be better alternatives that exist now.  Please let me know in the comments.

inc-num-region

This utility operates on a region of text, incrementing the number it finds on each line.

(defun inc-num-region (p m)
  "Increments the numbers in a given region"
  (interactive "r")
  (save-restriction
    (save-excursion
      (narrow-to-region p m)
      (goto-char (point-min))
      (forward-line)
      (let ((counter 1))
        (while (not (eq (point)
                        (point-max)))
          (goto-char (point-at-eol))
          (search-backward-regexp "[0-9]+" (point-at-bol) t)
          (let* ((this-num (string-to-number (match-string 0)))
                 (new-num-str (number-to-string (+ this-num
                                                   counter))))
            (replace-match new-num-str)
            (incf counter)
            (forward-line)))))))
For example, if you ran this command on a region consisting of:
foo0
foo0
foo0
It would change to:
foo0
foo1
foo2

Note: Reddit user rcfox has informed me that you can do something similar with a standard emacs regex replace with the special "\#" token

 add-code-review-note

Before we started using Review Board at my company, we would do over-the-shoulder reviews.  The add-code-review-note was a utility that I wrote to make it easier to take notes during these reviews.
(defun add-code-review-note ()
  "Add note for current file and line number"
  (interactive)
  (let ((file-name (buffer-file-name))
        (file-line (line-number-at-pos)))
    (switch-to-buffer-other-window (get-buffer-create "NOTES"))
    (goto-char (point-min))
    (when (not (search-forward "-*- mode:compilation-shell-minor"
                               nil t))
      (compilation-shell-minor-mode 1)
      (insert "-*- mode:compilation-shell-minor -*-\n\n"))
    (goto-char (point-max))
    (if (/= (current-column) 0)
        (newline))
    (insert file-name ":" (number-to-string file-line) ": ")))
This function adds notes to a buffer called "NOTES" (which it creates if it needs to).  The nice thing about this utility is that it puts the buffer in compilation-shell-minor-mode and prints out file and line number information in a way that this mode can interpret.  With this minor-mode enabled, file and line information is linked/highlighted and it's easy to jump right to the location in question.

Here is an example of what the NOTES buffer might look like:
-*- mode:compilation-shell-minor -*-
/home/jdavis/test.el:41: bad function name
/home/jdavis/test.el:55: unnecessary code
/home/jdavis/test.el:65: there must be a better way to do this
/home/jdavis/foo.cpp:11: maybe factor out into a separate function?
I have this function bound to "ctrl-c r".

automatic include guards

In C and C++, include guards are a necessary annoyance.  However, with this utility, I don't really have to think about them anymore.
(defun get-include-guard ()
  "Return a string suitable for use in a C/C++ include guard"
  (let* ((fname (buffer-file-name (current-buffer)))
         (fbasename (replace-regexp-in-string ".*/" "" fname))
         (inc-guard-base (replace-regexp-in-string "[.-]"
                                                   "_"
                                                   fbasename)))
    (concat (upcase inc-guard-base) "_")))

(add-hook 'find-file-not-found-hooks
          '(lambda ()
             (let ((file-name (buffer-file-name (current-buffer))))
               (when (string= ".h" (substring file-name -2))
                 (let ((include-guard (get-include-guard)))
                   (insert "#ifndef " include-guard)
                   (newline)
                   (insert "#define " include-guard)
                   (newline 4)
                   (insert "#endif")
                   (newline)
                   (previous-line 3)
                   (set-buffer-modified-p nil))))))
This hook automatically adds an include guard to new header files based on the file name.  So, if you create a new file called foo-bar.h, emacs will automatically insert this:
#ifndef FOO_BAR_H_
#define FOO_BAR_H_

// puts cursor here

#endif

next-file-with-basename

It's a fairly common idiom in C and C++ to have header files and implementation files with the same basename, but different extensions.  Like foo.h and foo.c, or bar.h and bar.cpp.

It's very useful to be able to jump between these files quickly, so I made a utility to make it easy to jump between files in the same directory with the same basename.
(defun next-file-with-basename ()
  "Cycles between files with the same basename as the given file.
   Usefull for cycling between header .h/.cpp/.hpp files etc."
  (interactive)
  (let* ((buf-file-name (replace-regexp-in-string
                         "^.*/" ""
                         (buffer-file-name)))
         (current-dir (replace-regexp-in-string
                       "[a-zA-Z0-9._-]+$" ""
                       (buffer-file-name)))
         (no-basename (equal ?. (aref buf-file-name 0)))
         (has-extension (find ?. buf-file-name)))
    ;; If the file is a .dot-file or it doesn't have an
    ;; extension, then there's nothing to do here.
    (unless (or no-basename (not has-extension))
      (let* ((basename (replace-regexp-in-string
                        "\\..*" ""
                        buf-file-name))
             (files-with-basename (directory-files
                                   current-dir f
                                   (concat "^" basename "\\."))))
        ;; If there's only 1 file with this basename, nothing to
        ;; do
        (unless (= (length files-with-basename) 1)
          ;; By making the list circular, we're guaranteed that
          ;; there will always be a next list element (ie. no
          ;; need for special case when file is at the end of
          ;; the list).
          (setf (cdr (last files-with-basename))
                files-with-basename)
          (find-file (cadr (member (buffer-file-name)
                                   files-with-basename))))))))
I have this function bound to "ctrl-c n"

Note: Reddit users davexunit and slavy pointed out a builtin function ff-find-other-file does something very similar to this.

in-line macro expansion

The last utility that I'd like to share is one that I wrote to escape the tedium of writing small snippets of boilerplate code.

Boilerplate code never bothered me much until I learned lisp.  Once you have something like lisp macros available, it's harder to stomach having to type very similar code snippets over and over again in other languages.

Since most languages (like C and C++) aren't homoiconic, there's no real chance of something like lisp macros helping us out here.

Something that would save me some time though would be if I could write shorthand s-expressions in my C and C++ code and have them automatically expand into the boilerplate code as I type.
(defun j-newline-and-indent ()
  "Same as \"newline-and-indent\" except it also expands 
   c-macros if it sees one."
  (interactive)
  (if (and (equal (char-before) ?\))
           (macro-function (car (preceding-sexp))))
      ;; This is a c-macro
      (expand-c-macro-in-place)
    (newline-and-indent)))

(defun macro-function (name)
  "Given a name, returns the c-macro-name symbol if it 
   exists as a function"
  (let ((macro-sym (intern (concat "c-macro-"
                                   (symbol-name name)))))
    (if (fboundp macro-sym)
        macro-sym
      nil)))

(defun expand-c-macro-in-place ()
  "Given that point is at the end of a c-macro, expands
   it in-place"
  (let* ((sexp (preceding-sexp))
         (macro-name (car sexp))
         (replacement-text (apply (macro-function macro-name)
                                  (cdr sexp)))
         (jump-to (string-match "!!!BODY!!!;" replacement-text)))
    ;; Delete macro invocation
    (backward-list)
    (let ((start-del (point)))
      (forward-list)
      (kill-region start-del (point))

      ;; Insert macro expansion and indent appropriately
      (insert replacement-text)
      (indent-region start-del (point))
      (when jump-to
        (search-backward "!!!BODY!!!;")
        (kill-line))))
  (c-indent-command))
This code replaces the normal "newline-and-indent" keybinding of ctrl-j with "j-newline-and-indent".  The j-newline-and-indent function looks for what I call a c-macro before the cursor and expands it in place if found.  Otherwise, it falls back to the normal newline-and-indent behavior.

In order for this to be useful, we need to define some c-macros.  These are just normal elisp function that start with "c-macro-".
(defun c-macro-doit (container-type arg1 &optional arg2)
  "Emits code for iterating over an stl (or stl-like) structure"
  (let ((iterator-name  (if arg2 arg1 "it"))
        (container-name (if arg2 arg2 arg1)))
    (format (concat "for (%s::iterator %s = %s.begin();\n"
                    "     %s != %s.end();\n"
                    "     ++%s)\n"
                    "{\n"
                    "   !!!BODY!!!;\n"
                    "}\n")
            container-type
            iterator-name
            container-name
            iterator-name
            container-name
            iterator-name)))

(defun c-macro-api-fn ()
  "Emits code for wrapping an api function in a try/catch block"
  (concat "try\n"
          "{\n"
          "   !!!BODY!!!;\n"
          "}\n"
          "catch(const std::exception& e)\n"
          "{\n"
          "   TRACE(\"Unhandled exception in function %s: %s\\n\",\n"
          "         __func__, e.what());\n"
          "   return -1;\n"
          "}\n"))
These are just 2 examples of c-macros.  As you can see, it's pretty easy to add new ones.
(doit std::vector<int> myContainer)
// at end of the previous line, hit C-j and this expands to:
for (std::vector<int>::iterator it = myContainer.begin();
     it != myContainer.end();
     ++it)
{
   // cursor goes here  
}

// You can also specify the iterator name if you want
(doit std::vector<int> myIt myContainer)
// expands to
for (std::vector<int>::iterator myIt = myContainer.begin();
     myIt != myContainer.end();
     ++myIt)
{
   // cursor goes here
}

(api-fn)
// expands to
try
{
   // cursor goes here
}
catch(const std::exception& e)
{
   TRACE("Unhandled exception in function %s: %s\n", 
         __func__, e.what());
   return -1;
}

Note: Reddit users aaptel and stack_pivot informed me about yasnippet which looks like it does this and a whole lot more.

Conclusion

So, there you have it.  Those are some of the utilities that have stood the test of time for me.  They help define the shape of the pretty little bush that is my .emacs file.

What are some small utilities that you find indispensable?  Let me know in the comments.