added documentation
This commit is contained in:
parent
1b1ef203f0
commit
b1597047f1
1 changed files with 63 additions and 0 deletions
63
README.txt
63
README.txt
|
|
@ -22,3 +22,66 @@ port=11573
|
||||||
And you're done, it's installed
|
And you're done, it's installed
|
||||||
you can run it by running my facetracking.sh shell script which applies some settings I found useful for my camera (though I'm unsure how compatible they are with other setups), or you can just use 'python3 facetracker.py' after re-activating the virtual environment
|
you can run it by running my facetracking.sh shell script which applies some settings I found useful for my camera (though I'm unsure how compatible they are with other setups), or you can just use 'python3 facetracker.py' after re-activating the virtual environment
|
||||||
|
|
||||||
|
|
||||||
|
|
||||||
|
-----Using the software-----
|
||||||
|
Generally you should be able to run it and just open the network webcam in VTS
|
||||||
|
I don't remember if there are other steps
|
||||||
|
|
||||||
|
but here are some things I've noticed that should make you have an easier time
|
||||||
|
*Running resolutions above 480p doesn't seem to improve tracking and causes a lot of latency with the webcam
|
||||||
|
*Running at higher resolutions might work if you use a lower framerate like 24fps, buy it's not worth it imo, your face gets downscaled to 224x224 before getting sent off to tracking
|
||||||
|
*Lighting largely doesn't matter from my testing (this was a goal of mine)
|
||||||
|
*Idk how well my automatic brightness works with darker skin, but if it's an issue you can edit the targetBrightness variable in facetracker.py
|
||||||
|
*30fps works great, but if you can somehow feed this more *I think* it can do more, based on frame times it should be able to do 45-50fps, but my webcam doesn't go that fast
|
||||||
|
*you can pass the --preview 1 flag to get a preview window, this shows you the webcam feed after having the brightness adjusted
|
||||||
|
|
||||||
|
|
||||||
|
-----Changes from Openseeface(in no particular order)-----
|
||||||
|
|
||||||
|
-Major restructure
|
||||||
|
-I've broken out major chunks of functionality into separate files to make them easier to deal with and read
|
||||||
|
-I've tried to break functions down into multiple smaller parts when possible
|
||||||
|
-I've taken a more object oriented approach in some places
|
||||||
|
-I've removed features that didn't fit my needs
|
||||||
|
-Removed functionlaity for tracking multiple faces
|
||||||
|
-Removed options I didn't forsee myself using (this was intended to be just for me)
|
||||||
|
-Added more threading and multi-processing
|
||||||
|
-The webcam (and some new image processing) are now on a separate process
|
||||||
|
-Previews are now a separate process because I didn't want to deal with the related performance weirdness
|
||||||
|
-Console messages are now handled via a helper thread, idk if it does any good, but it can't hurt
|
||||||
|
-VTS communication is also handled via a helper thread
|
||||||
|
-Added image pre-processing
|
||||||
|
-I'm now applying a gamma curve to the webcam output to make faces more visible
|
||||||
|
-The gamma curve is calculated after face tracking is done, it uses a copy of the webcam frame and face location data so get the average brightness of the face
|
||||||
|
-This has drasitcally improved low light performance
|
||||||
|
-It also lets me run the webcam with no gain
|
||||||
|
-Removed the separate library that handled the webcam, then broke that off into a separate library again
|
||||||
|
-The stuff around the webcam is still simpler now, most of the existing cases didn't apply to my use case
|
||||||
|
-Revamped the way features are calculated and normalized
|
||||||
|
-This was kind of my original intent, my eyes kept registering as closed when they were not
|
||||||
|
-Removed features that didn't seem to be used by Vtube Studio
|
||||||
|
-Added functionality to apply response curves to features
|
||||||
|
-Removed the calibration period in favor of a system where the limits of features slowly decay towards a center point
|
||||||
|
-Removed the average from the way featured are normalized
|
||||||
|
-Added a feature to prevent errant eye movements
|
||||||
|
-Each eye has an average confidence and standard deviation calculated every frame
|
||||||
|
-eye movements more than two standard deviations below average are severely restricted
|
||||||
|
-The restriction is based on how far the confidence is from the cutoff point
|
||||||
|
-Added early exits to situations where subsequent steps will fail
|
||||||
|
-The main process skips frames when the webcam falls behind, once webcam latency is long enough to be late it will remain behind otherwise
|
||||||
|
-Added warnings to various states such as late webcam frames, longer than ideal frames, and early exits
|
||||||
|
-Changed which stats are shown on exit, added some stats that get tracked like webcam latency
|
||||||
|
-Added conditions to skip steps when they are unnecessary, such as not rotating the eye images for small angles
|
||||||
|
|
||||||
|
|
||||||
|
-----Credit-----
|
||||||
|
For documentation on how openseeface works and the computer vision models, go see https://github.com/emilianavt/OpenSeeFace I don't want to leave documentation here that makes it seem like this is all my work
|
||||||
|
|
||||||
|
Seriously, emilianavt did all the hard work
|
||||||
|
|
||||||
|
-----License-----
|
||||||
|
idk, I didn't plan to get this far
|
||||||
|
feel free to use my software and code for whatever, it'd be nice to get credit if you do
|
||||||
|
Just don't do anything to get me sued, or harass people, or make hateful content
|
||||||
|
Do use my software to be
|
||||||
|
|
|
||||||
Loading…
Add table
Add a link
Reference in a new issue