[Nuke]

a thread over on the btrl board got me thinking about writing a suite of programs that would essentially allow a pda, tablet, ect, to be used as an input/output device. and i was curious if it would be possible to integrate such an interface into the freespace engine.

how it works is a host application would run on your computer, it would act as a proxy between any game and the pda. the game could send raster or vector graphics to the host app, the data would be formatted, compressed and sent over a network connection to the pda. a client app on the pda would take that data and draw/render it onto the touch screen. also any input functions from hardware buttons, the touchscreen, or from on screen gauges, knobs, sliders, ect are sent back to the host app, where the data can be passed to virtual joystick (ppjoy), mouse(?), or keyboard(?) drivers, or by direct interface with the game.

what i want to develop is pretty much a universal app that can be used in games in general, not just freespace. it would be open source and hopefully could be ported to other platforms. its also important that it be compatable with as many games as possible. and would allow for custom generic panel profiles to be used, in case adding interface code to the game engine is not possible. these profiles would allow for custom layout of animated interface objects with a set background image. the interface objects can be mapped to keystrokes, mouse or joystick events, so you could still create a usefull panel for whatever game you want.

however i still want to be able to have an option for a shared memory space with the game, whether it be through the engine itself or through lua scripting. this would allow for scripted gauges, or rtt-ed surfaces to be used. idea would be an external radar display, turret view and control, perhaps control of lateral thrusters or an interactive com display. the possibilities are endless.

theres more technical stuff here
http://www.game-warden.com/forum/showpost.php?p=87695&postcount=31
http://www.game-warden.com/forum/showpost.php?p=87723&postcount=34
http://www.game-warden.com/forum/showpost.php?p=87739&postcount=35

[WMCoolmon]

I think your best bet would be to do something that relies on TCP/IP networking. That way, as long as you have a network connection to the item, you can write a program that will read the data being sent to the device. (Or you could redirect it to localhost for an internal device eg secondary screen)

The chief problem is that not everybody has a PDA and you have to make it useful to use. I'm not sure what benefit a touch screen would provide you when you already have 100 keys on the keyboard. You could use sliders and stuff, but it would often be faster to pound page up, for instance.

Basically, I think this would be much more useful for non-game applications, such as DVD players. If you were to integrate the host app with the Win32 API, for instance, you could set it up so that the app could be fed predefined templates for button locations on the DVD control panel window, and simulate mouse presses on those locations. Those could then be routed to the PDA, which would basically function as a remote.

Still, I wonder how a universal interface would be possible? PDAs have all shapes and sizes of screens. In order to code something truly crossplatform you'd have to account for that, or create templates for several generic PDA screen types. You'd also have to design some good easy-to-use interface design tool(s), so people could quickly get into creating the interface for games and whatnot.

There's also the matter of what language to do stuff in; C++ may work for Windows CE, but I doubt it'd be very easy to port it to maemo (linux-based). In order for this to be accessible, you'd have to have it be easy to install on any PDA - no mucking about with installing extra graphical libraries, if possible.

So for bandwidth concerns, maybe have an application template which contains PDA templates. The PDA templates depend on object templates which defines buttons and so forth. Meanwhile the host portion of the application template determines how the host app is going to interact with the target app. The appropriate PDA template and object templates are sent to each client. That way, communication can be reduced to basic action and data rather than sending vector graphics etc every time the page changes.

It sounds pretty interesting though. I'm willing to help out with it, though I can't promise too much in the way of raw coding.

[Deimos]

I must say that i absolutely love the idea. It would be nice to put my simpad (800x600 screen - that's a lot of free space ) into more use than just an ebook reader

I'm not sure what benefit a touch screen would provide you when you already have 100 keys on the keyboard. You could use sliders and stuff, but it would often be faster to pound page up, for instance.

Well, i think that's not entirely true - just think of an dynamic clickable escort targets list, or escorted ship's attackers, sorted by their distance from their targets. And sliders on the touchscreen also have some appeal (Star trek control panel on your desk )

Still, I wonder how a universal interface would be possible? PDAs have all shapes and sizes of screens. (...)
There's also the matter of what language to do stuff in; C++ may work for Windows CE, but I doubt it'd be very easy to port it to maemo (linux-based). In order for this to be accessible, you'd have to have it be easy to install on any PDA - no mucking about with installing extra graphical libraries, if possible.

Well, there are some screen sizes available (from 320x240, 640x240, 640x460, even up to 800x600 in some HPS's), but if graphics would be vector rather than raster - based (for eg. radar view), rescaling shouldn't be much of an issue. But of course some custom layout generator would be useful as well to optimize use of screen area.
For platform portability, there is SDL. I've been able to recompile few of my SDL win32 programs almost without any modifications and run them on windows CE 4.0, and WM5 devices i just had to put SDL.dll file in program's dir. It should work with linux-based ppc's as well. And as a bonus, it would be portable to desktop OS's as well (you could use a laptop as a MFD), and theoretically even to Symbian (however it would be a bit if your MFD started ringing )

Anyway, the idea is really great.

[Nuke]

i see no reason why the client app has to run on a pda, theres really no reason it cant be ported to other platforms. so long as the interface over the network is the same and the client app does the same job. the client app could be ported to java, c, c++, vb, lua, whatever. i could do a version for palm, pocket pc, or linux distros for pda and maybe cell phones. also i could have an x86 version for laptops, tablets, micro pc, which could be compiled on windows, linux, mac. those would be easyer to write actually because they would use the same codebase for the most part. the portables will be the hard part.

so far i have a few pcs, a laptop, an ipaq 3600, and an 5th gen ipod running rockbox (with hacks its possible to run an interface app with replacement firmware like ipod linux or rockbox through a serial connection). i could probibly get a surplus palm for real cheap, assuming i can find one with a network interface. as for operating systems, i have several versions of windows and some popular distros of linux on hand. so those are the platforms i will work with initially.

il probibly be using c++ for the most part. im still doing initial research into what libs i want to use. on the client side i really just need to be able to render graphics. i was looking at gapi, but id perfer to use sdl if i can get it to compile. supposedly theres support in the library's unstable branch for windows ce. but i will have to evaluate it better. ive never written network code before so any api suggestions are welcome.

i want support for either vector or raster images, and possibly both. vector would be used for low end devices and slow connections. for raster im probibly gonna use standard dxt formats, mainly dxt1, 5'6'5,' 8'8'8 and a couple other custom formats, like the 12 bit compressed format based roughly off dxt1. part of the init will be the host asking the client what it can do, client replies with info about its networking and graphics capabilities. then the user can set the options, like what resolution to use, what pixel format to use and how fast to send control updates and load a profile.

as for other uses, there would really be no limit to it. one of my ideas was to use it as a throttle quadrant, so i could set mixture and prop pitch for multiple engines. an idea of maybe creating an orbiter module to display mfds from the sim. the primary functions will be mainly for flight simmers. but if i can get mouse, keyboard and joystick to work, it would pretty much cover all the bases as far as input is concerned. figuring out how to implement virtual drivers for those devices on pultiple platforms will be another main challenge. the only virtual driver i know off the bat is ppjoy for joysticks. its windows only and closed source, but it does come with code to demonstrate how to access the driver. it supports 8 axes and 16 buttons.

[Deimos]

As far as i know, SDL can be used along with GAPI (would be nice to have on older devices without direct framebuffer access, like CE3 and CE4), for example ScummVM uses both (www.scummvm.org) - maybe looking trough its source would help?
And for keyboard emulation, there's always a windows program called joy2key that could translate ppjoy outputs into keystrokes and mouse movement if you can't figure anything else right away - http://www.electracode.com/4/joy2key/JoyToKey%20English%20Version.htm

[Nuke]

why use both? sdl can do pretty much everything gapi can do. sdl has the added bonus that ive used it before and am very familiar with its ins and outs.

i was thinking of going with ppjoy output exclusively and let users have a 3rd party control converter if they wanted mouse or keyboard input. it does limit you if you want to use another platform for the host app. on the other hand im sure everyone would love to have open source virtual drivers that can be used on multiple platforms.

[Swifty]

Nuke, you should look into TouchBuddy. It's basically an app where you can use a touch screen as a hotkey input device for your games. It's really useful for flight simulators.

http://www.youtube.com/watch?v=oKv8KR8bX7w
http://www.touch-buddy.com/

You can either connect the touchscreen directly as another monitor on your computer or TouchBuddy can do WMCoolmon's suggestion of sending input data through TCP/IP.

Unfortunately, it's not open source nor does it have any capability of outputting live data (gauges, instruments) through the touchscreen though rumor has it the developers are now trying to work on that feature.

EDIT:

I also forgot about PocketMFP as well (http://www.pocketmfp.com/). Though, it only works with Falcon 4.0 and with PocketPCs.

[Nuke]

yea, ive seen those, thats why i want to develop this one