[iadesdragon]

Wouldn´t it be cool to make FSO see your head? I Know we have TrackIR support. But Face API simply tracks your Face... using the ( build in ) Webcam you allready have.

As far as i understood their offering, FaceAPI is free for non-comercial use....

Any of you hard core programmers willing to give it a try?

http://www.seeingmachines.com/product/faceapi/

[Nuke]

trackir framerate = 120
webcam framerate = 30

youre better off with a wiimote+freetrack than cheap webcam.
or since trackir 5 came out 4s are pretty cheap and still offer 6dof.

[Dragon]

I'd still like to be able to use webcam instead of TrackIR.
If somebody wants to use TrackIR, OK, but I'd like to be able to have face tracking without spending a cent.

[chief1983]

His point is performance would probably be sub-optimal with a crappy webcam, even if it 'works' with it.  You'd at least want to get a halfway decent webcam.  Plus, FreeTrack supports webcams too and should already work I think.  But you still want a quality cam, preferably 60fps at least.

[Nuke]

i dont think they even make 60 fps webcams. the point of a webcam is to produce a small image that can be easily moved even over low bandwidth connections, so the performance is intentionally kept very low. im sure they exist, but your probibly gonna pay for it (to the extend that your better off just getting special purpose hardware). the dot based algorithms are a lot faster than the face tracking ones, and it doesnt require much processing since most of the work is done with a light filter. faster means less lag in the input.

freetrack support for webcams isnt bad though and sometimes 30fps is enough, though i find the low response time can be somewhat nauseating. i find cheaper webcams without a whole lot of extra processing work best (the trackir site keeps a list of whats good and whats not), and for an ir filter floppy disks work well, get about 3 ir leds and rough em up with sandpaper. figure out what their forward voltage is (usually between 1 and 3) and find a power source (old 3.7v cellphone batteries work nice) hook the leds in paralell and use a voltage divider if needed to keep from giving the leds too much juice. then you need to build some kinda frame to support them. if you use a wiimote try to space them out as far as possible. though the wiimote kinda has a lower than necessary field of view so it needs to be further away, but its ir camera is very sensitive and can work across a room if need be. getting the wiimote to talk to bluetooth is the hard part, but almost any bluetooth stack other than the one that comes with window works.

[iadesdragon]

What i found tempting with this approach....

it (seems to ) be free, and you do not need ANY targets at all. SO this would work werever you go... ( as long as there is a webcam )

[Nuke]

but you have to not only put up with a low refresh rate but also an extended lag time to process the data. face tracking algorithms require checking multiple consecutive frames to get an approximation of what a face is and determine how to track it. it needs to find a number of distinctive points, usually eye glare, tip of the nose, chin, dimples ect. point is it doesnt know what its looking for so it needs to check more frames and find spots that change the least. then it can establish a model and track it. or course if the lighting changes or somone else uses it the system has to re-adjust and come up with new points to adjust by. allowing the camera to auto-adjust lighting levels, exposure, white balence, ect almost always cuts your framerate and requires processing time.

with targets and filters you only need to do some basic pixel math to isolate and threshold points and since the pattern is predefined and uniform you only need one frame of data to get all the info to compute a transform matrix. usually you set your exposre to as short as possible but long enough to get the points. but when it comes down to it the physical infrared filter does 90% of the work and you get your angles faster.

now you really want a face tracking api you probibly dont have to even integrate the api into the engine. i know i seen at least one face tracking proggie that output to some kind of mouse control or ppjoy interface.

[Goober5000]

I wonder if we should institute a rule that anyone requesting a SCP feature has to have more than a certain number of posts.

[Nuke]

i wouldnt say that the idea is totally bogus, i just think there are better alternatives.

[Goober5000]

Oh I definitely agree with that.  But more importantly, I'm just wary that this is a request from a new user, who may not stick around on HLP.  If we go and spend the effort needed to code this FaceAPI feature, is he going to be around later to use it?

[iadesdragon]

Hey Guys,

this has been more like Thinking Aloude, not a feature request at all. I only wondered if this might be a kewl thing because there is no soldering ant thinkering with your web cam involved...

I have freetrack runing all right and just got myself a trackir... but i also saw ( while seting up the stuff ) that a lot of people have problems getting the LED stuff work or do not like to spend a lot of money for this...

Besides you are right in some respect, especally when it comes to frame rate. The pure math power should not be a big deal with a up to date machine.

Anyhow, im more of the embedded systems guy, so ask me if you like to interface a 6DOF Moving platform or real usable hardware MFD`s.

And, while this account is new, im with Freespace and altough FSO for years now, only i`v been off from active use due to dipper duty for quite a while and simply did not recall wich original account i used :-)

BTW. unbeliveable grate work! A lot has been fixed, enhanced, since i last used FSO...



Rgds...

[Nuke]

Anyhow, im more of the embedded systems guy, so ask me if you like to interface a 6DOF Moving platform or real usable hardware MFD`s.

funny you mention that. ive been playing with arduino for like the last 3 months. ive also written a lot of serial code lately, for a variety of reasons. did an interface program for ppjoy so i could send serial packets containing joystick info, which i captured from a number of various devices. also managed to control servos from a serial terminal. i was actually thinking of doing a generic scriptable serial interface of sorts so that freespace would have some facility for operating external hardware, mfds, and so on. alternatively some kind of shared memory interface that would allow fs to talk to some other program on the system (which in turn could talk to hardware). win32 serial interface seems to essentially involve a file handle so it might be possible to open a com port through the existing cfile interface. my duemilanove seems to be capable of about 230400 baud, which is enough for operating mechanical devices like servos, or transmitting some small amount of data to an mfd.

[iadesdragon]

Nuke,

i`m with you! Must be mind melt :-). That is exactly what I was thinking about. Having a interface to Freespace like today is quit common with FSX or X-Plane. I just started doing hardware for an MDF some days ago (not especially for FSO however) but would really be happy to have someone on the PC / MAC side assisting in doing the Host side coding.

Because i believe having a usable hardware cockpit with MDFs ans switches and such, probably even combined with a "simulator chair" that at least ads pitch / roll and yaw to the experience would REALLY add to the overall experience of the game.

However, i need to find a way to have FSO send the critical data to USB or ( maybe easier and better ) an local IP A dress ( which again is a embedded system )

Envision something the size of a a deckchair with a 32" Monitor mounted in front of the Gamer underneath this monitor you got 2-3 4-5" LCD MDF with touch and a bunch of buttons all around.Add some meters and switches as liked. Then Equip this with a TrackIR and something like the Logitech G940....

now simply screw down your Computer on the back of this chair (serving as a counter wight as well...

Since all Coponents are mounted to this very chair only thing that needs to be supplied to it is mains. THis allows for the chair for infinite rotation in Z while it chan lean in X and Y by aprox 45°....

I Myself have like 20 years of experience in Embedded Controll of fast production machines beeing the principal engineer on several projects that involved a lot of fast and precise movement.

In my privat lab i am able to do Layout and have PCB done, i can design even complex 32 bit embedded systems with graphics, Ethernet and such...

Additionally there would be the possibility to do organize and manage hardware build runs for people interested in building one ( because i know that most people do not like to solder SMD stuff nor have experience doing it)

So all we need is a Team and enough people being interested in doing the work, moding the ships etc...

But, this would be a Huge Project lasting several month if not years... ( while some of the stuff could be used from other projects, besides the really cool stuff most often is not open source )

So if there are enough People interested (we need quit some units to build to keep it cheap) and there are some willing supporters...

(ANd people that may live with my somewhat imperfect English due to me being native German :-) )

Rgds,

[Nuke]

thats about the kinda thing i was thinking. for communication i was initially thinking of doing a serial interface, of couse in a game engine it might be advisable to run serial code in another thread. mt is something ive never done before (unless you consider stuff i did with my lego rcx ), but i here freespace uses it for specific things such as input and i may be able to use that as template. ip or usb is definitely a possibility, especially ethernet (theres an arduino shield for that). i like arduino because its cheap, expandable, and very easy to get into even without any ee experience. but the idea is to make the game side interface scriptable with lua so that it is as versatile as possible. the game really only needs to read and send packets to a micro controller and so long as the uc can decode the packets and use them then pretty much anything goes.

a full scale simulator is kind of out of my budget, though i was thinking of a scale model "smurf simulator" made from r/c components as a proof of concept. so far i have my arduino duemilanove and 2 arduino-in-breadboard setups, all running atmega328ps and was thinking of setting up an i2c network. the arduino board would kinda be used as the master device and would talk to the computer while the other two would essentially be slaved to it and essentially control servos, motors, and read sensors. assembally of the smurf sim is a non-issue, so right now its mostly a programming job.

also forgive my bad english (im american)

[iadesdragon]

Ya, did some thinking today...

Basically i believe using one of the new 32bit PIC micros would be fun... they have a reasonable horspower of aprox 120 MIPS and feture a lot of nice interfaces including 10/100 MBit ethernet.
I altough thought it would be nice and hady to have this in one board.

So all you need is a power supply ( i would recomend a of the shelf one because we ar hobists, besides i even could design one) and a ethernet cable you plug into your PC or Switch.

The PIC is abprox 10$, and would give half a MegByte of Flash along wit a good quater of a meg RAM. What i am thinking about now is how one would layout a cokpit for a space age fighter. Shure no old style handed instruments, those wouldn´t work in space. One big LCD? to costly....

what i think might be fun is 3 seperate touch screen LCDs that can be configured as liked... these can show all that is shown arround the main hud ( comm menu, weapon selection, tagret screen etc.. ) and you can e.g. change the pwer system settings by sliding your finger...

Only downside i can think of right now... not eough computional power to do other than wire frame models for the target display. and the models have to be imported to the panel first..

i really thing one 4.3" LCD in center acompanied by two smaler (3.5"? or 2.4") LCDs left and right would be a good starting point...

These would be aprox 120$ total if you buy them single unit wise at newhaven displays. add the board an switches gluelogic and some 2D grapichs controllers...

Bet 250$ something for the full Cockpit-Kit ( plus you have to build something to go allong as a housing yourself.

About the "rocking chair", figure 500$ should do for the system parts 100$ each for x and y drives another 100$ for yaw drive ( optional ) 100 bugs for the Electronics....

so if you put the PC aside a total kit including something like the logitech G940 would cost within 1500 to 2000$ i would guess from first planing, and depending the amount of kits you build ( if there are enough people intrested would be a great thing)

Biggest development herein is, as always, software...

About i2c, generally easy to use but only few OTS parts you can use for something like that ( two if i recall right a D/A A/D GPIO combi unit and a 8Bit Parralel IO ) both of them quit costly...

But, as i said, i think this only can be done in a team efford and if it is the EE side, count me in on that...

I`ll do some sketches tomorrow to give a better vision on what i thing about...

Lets do a first Layout, User Interface is what we ar talking about... once thats done... we can talk hardware :-)

So however is intressted input please!

WHAT WOULD A SPACE AGE FIGHTER COCKPIT LOOK LIKE?

[iadesdragon]

Maybe we need a new subject ;-)

 

[Nuke]

yea that too, maybe some moderator will come and split the topic

mini-lcds are rather overpriced and if you dont mind having an analog interface you can get 3.5" 720*480 lcds directly from china for around 30 bucks. i so happen to have one right in front of me, its a little bit too small for an mfd though. larger screens can be really cheap if you look for broken portable dvd players (90% of the time its the dvd player thats shot and the screen is ok), again analog interface (probibly also @ 720*480).

either way most modern video cards have at least 1 analog display port that takes some kind of splitter cable, so you might be able to run graphics right off of the video card from a program on the computer. if you know how to do some signal processing you could do some kind of frame multiplexing to transmit 3 or 4 different video sources over the same analog line and display them on separate screens.

the mcus may not be able to do much in terms of rendering, but i bet you could use dxt1 compression (4 bits per pixel 5'6'5 but requires dimensions in multiples of 4), to allow digital transmission of images. at 720*480 its about 172k per frame. serial wouldn't cut it but ethernet or usb could do it just fine. the mcu would have to decompress the data and forward it to an lcd driver or signal generator of sorts.

[Herra Tohtori]

Well, I have found 30 frames per second to be sufficiently accurate for FreeTrack purposes in IL-2 Sturmovik - with a carefully set up profile to adjust turn rates, damping and smoothing. The response time is fast enough that there's no perceivable delay, and you can keep the view static enough that aiming and performing other accurate maneuvers is very much possible.

Granted, the program (FreeTrack) interpolates it into 120 FPS with a multiplier.

However, for a computer it's much easier to initialize a plane using three dots as reference points than a flexible, shape-changing object like human face, and that's definitely going to increase the amount of cycles that it takes to decide the perceived position of the head at any given time.

[iadesdragon]

Big advantage of the somewhat more expensive mini-lcds is, you can buy them on a regular basis, you get full support by the Distributor etc.

Problem is that most LCDs have some specialties with them so you can not easily exchange them in a design, therefor using analog interfacing on the other hand might seem a advantage, however i usually found those ( dvd player ) PAL or NTSC Analog Displays ( especially the China style ) to give a unsatisfactory crappy image.

I think, with all the afford required to develop & build something like this, the outcome should be pleasing and especially in a way that it ( we are talking open source i believe ) van be re-build easily by others. Using ebay parts proved doom to quit a bunch of projects in the Open Hardware domain already. 

But, as it seems, there is not to much interest in such a cockpit or nobody found our topic by now ;-).

Good thing however is that we do not have to reproduce a "real" cockpit as the Sim Builders do therefor there are quit some options.

Another possible approach would be to use a reasonably sized OTS LCD Panel with DVI ( lets say 15-17" and only design a Sliders and Touch Panel overlay that one monts directly in front of it.

This unit only as cutouts where we like to have MDFs so only the used areas of the LCD are visible. PC/MAC does all the stuf rendering on the secondary display ( The LED Pannel )

The Switch and Touch Elektronik board only does emulate kind of a keyboard connected via usb...

Is there a way to tel FSO to render the secondary HUD elements to a seperate View?

THis would speed development and also cut down cost.

Quick look to the bay shows 15" 1028 x 768 DVI is a cheap buy.

 

[Nuke]

ive had luck implementing joystick axes, sliders, and buttons through the ppjoy joystick interface. i read all axes and buttons on the arduino, format the data into a serial frame. im using a simple frame with 170 as the sync byte, followed by an x number of bytes for buttons (1 bit allocated to each), followed by packed 10 bit joystick data (0 padded at the end) and a 1 byte checksum.

ppjoy just so happens to have a code sample for creating programs to talk to the virtual joystick driver. im already working on a (still rather buggy) command line program for reading a serial packet and reporting the axis and button states to ppjoy. that should cover input from the mfds, however you would probably need to mix in axes and buttons from the joystick from there (unless you cockpit had its own built in, which could send its axis data over the same interface), and ppjoy is kind of limited to i think 16 buttons and 8 axes and doesnt seem to support hats (not sure). another alternative is using glovepie, which could convert some of that data to other formats (keyboard, mouse, ect). needless to say freespace only uses 1 joystick.

if analog lcds dont cut it, there are a number of digital screens on sparkfun to choose from. since i have no experience using screens with micro controllers i was thinking something like this to get me started. however after glancing over the driver code im concerned that there wouldnt be much space on the mcu for much of anything else.

as for rendering, the freespace renderer only supports 1 display, and i doubt thats subject to change. however freespace does kind of support render to texture, if these dynamic textures could be sent to an external device, then it would be possible to have game rendered graphics on the mfd (of course im not sure what pixel format they would be in). it however wouldnt be anything as big as 1024x768.