Friday, August 31, 2012
Sunday, August 19, 2012
EyeTV with Commercial Skipping HD DVR Server
Jun 5, 2012 1:10 AM
I use EyeTV HD and Turbo.264 HD to record and transcode h.264 shows for playback on a Mac, Apple TV, iPad, iPhone. I also use a self-modified version of etvcomskip to detect and mark commercials, which are then automatically skipped at playback. This makes tv viewing a lot more pleasurable and efficient, and is especially helpful controlling advertiser access to children. In case this is useful to others, I've outlined the steps here:
- Go donate $10 or more at Eric Kaashoek's website Comskip and download his HD-capable Comskip software. "Commercial" commercial-skipping companies (ReplayTV,TiVo, Dish, etc.) are quickly sued out of existence or into submission, even if they simply provide a basic "30-second skip" feature. Comskip is open source beerware that you can download, install, and use yourself, and you can't sue anyone for publishing source code. And rather than providing a simple 30-second skip feature, it detects, marks, and provides that capability for automatic commercial skipping.
- Comskip a Windows executable, so you'll need WINE (WINdows Emulator) on OS X to run it. First, download and install Xcode (free from the Apple App store), then download and install and install Macports, then in Terminal, run "sudo port selfupdate; sudo port install wine-devel". Finally, make sure that /Applications/Utilities/X11.app is added to your list of applications to run at login (WINE uses X11 to display Windows windows). Yes, this is windows emulation, but on a half-decent box you'll see that commercial skipping runs at around twice real-time for h.264 HD recordings.
- Download and install etvcomskip (and PyeTV if you want EyeTV to work with Front Row). The 2010 version of etvcomskip doesn't work with Comskip's latest HD capabilities, and the latest Comskip mpeg libraries do not allow live tv processing, so install these files to get everything working. Once you done this, you'll have a commercial-skipping HD DVR box on your Mac that can stream to Apple TV, iOS, or export to iTunes.
Comskip software
This lives in the directory /Library/Application Support/ETVcomskip.
Edit the file ./comskip/comskip.ini to use these HD-tuned settings, tailored for US cable:
| detect_method=127 | ;1=black frame, 2=logo, 4=scene change, 8=fuzzy logic, 16=closed captions, 32=aspect ration, 64=silence, 128=cutscenes, 255=all |
| max_avg_brightness=21 | ;maximum average brightness for a dim frame to be considered black (scale 0 to 255) 0 means autosetting |
| non_uniformity=50 | ; Set to 0 to disable cutpoints based on uniform frames |
| max_volume=20 | ; any frame with sound volume larger than this will not be regarded as black frame |
| logo_at_bottom=1 | ; Set to 1 to search only for logo at the lower half of the video, do not combine with subtitle setting |
| intelligent_brightness=1 | ; Set to 1 to use a USA specific algorithm to tune some of the settings, not adviced outside the USA |
volume_slip=150
| output_dvrmstb=0 | ; Set to 1 if you're running DVRMS-Toolbox |
| live_tv=0 | ; set to 1 if you use parallelprocessing and need the output while recording |
| live_tv_retries=4 | ; change to 16 when using live_tv in BTV, used for mpeg PS and TS |
Edit the file ./MarkCommercials.app/Contents/Resources/MarkCommercials.py to use WINE to run comskip.exe nicely. Grep for these changes:
EyeTV.launch()
msg = 'Error: unable to communicate with %s\n' % options.app
#cmd = '"/Library/Application Support/ETVComskip/Wine.app/Contents/Resources/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "%s"' % MpgFile
# MacPorts 64-bit wine
#cmd = '"/Applications/Wine.app/Contents/Resources/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "%s"' % MpgFile
cmd = '"/opt/local/bin/wine" "/Library/Application Support/ETVComskip/comskip/comskip.exe" --ini="/Library/Application Support/ETVComskip/comskip/comskip.ini" "%s"' % MpgFile
# nice the wine command
cmd = "/usr/bin/nice -n 14 " + cmd
EyeTV software
EyeTV uses the triggered AppleScripts RecordingStarted and RecordingDone in the directory /Library/Application\ Support/EyeTV/Scripts/TriggeredScripts. Open these using the AppleScript Editor and use these scripts instead. You will probably have to Save As... on your desktop, then copy the updated script back into the correct directory to avoid permissions issues.
RecordingStarted.scpt
on RecordingStarted(recordingID)
delay 10
-- comskip81 uses ffmpeg and does not support live tv; put this in RecordingDone.scpt
-- set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --log " & recordingID & " &> /dev/null &"
-- display dialog cmd
-- set cmd to "env > /tmp/etv_test.log &"
-- do shell script cmd
--disable this if you do not want a logfile written
write_to_file((short date string of (current date) & " " & time string of (current date)) & "Recording Started run for ID: " & recordingID & (ASCII character 13), (path to "logs" as string) & "EyeTV scripts.log", true)
end RecordingStarted
on write_to_file(this_data, target_file, append_data)
try
set the target_file to the target_file as string
set the open_target_file to open for access file target_file with write permission
if append_data is false then set eof of the open_target_file to 0
write this_data to the open_target_file starting at eof
close access the open_target_file
return true
on error
try
close access file target_file
end try
return false
end try
end write_to_file
RecordingDone.scpt
-- Run the python MarkCommercials script for the given recording
-- this must be run with the RecordingStarted script
-- it will check if there were multiple PIDs for the recording and runs MarkCommercials for each pid
-- requires updated MarkCommercials which allows specifying the pid
-- by Ben Blake, September 2009
-- modified for latest Comskip, which cannot be run until after recording is finished; waits for Turbo.264 HD to stop running as well
-- S.T.Smith
global LogMsg
on RecordingDone(recordingID)
set ProcessName to "Elgato H.264 Decoder"
set DelayTime to 60
set MaxDelays to 4 * 60 -- four hours
-- delay until the process slows down or stops running
repeat MaxDelays times
delay DelayTime -- delay at least once to give it a chance to start
set pcpu to CPUPercentage(ProcessName)
if pcpu is equal to "" or pcpu < 2.0 then exit repeat -- break out of delay loop if it's not running
end repeat
delay 10
-- comskip81 uses ffmpeg and does not support live tv; take this from RecordingStarted.scpt
set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --log " & recordingID & " &> /dev/null &"
-- display dialog cmd
-- set cmd to "env > /tmp/etv_test.log &"
do shell script cmd
set LogMsg to ""
CheckMultiplePIDs(recordingID)
--disable this if you do not want a logfile written
if (count of LogMsg) > 0 then
write_to_file((short date string of (current date) & " " & time string of (current date)) & LogMsg & (ASCII character 13), (path to "logs" as string) & "EyeTV scripts.log", true)
end if
end RecordingDone
-- testing code: this will not be called when triggered from EyeTV, but only when the script is run as a stand-alone script
on run
tell application "EyeTV.app"
set rec to «class Unqu» of item 1 of every «class cRec»
my RecordingDone(rec)
end tell
end run
-- compute the percentage CPU used by ProcessName
on CPUPercentage(ProcessName)
set ProcessPS to do shell script ("ps -axwwc | grep '" & ProcessName & "' | grep -v grep || true")
if ProcessPS is not equal to "" then
set ProcessID to word 1 of ProcessPS
set ProcessPS to do shell script ("ps -xwwco pid,ppid,%cpu -p " & ProcessID & " | tail -1")
set ProcessCPU to word 3 of ProcessPS
return ProcessCPU as number
else
return ""
end if
end CPUPercentage
on CheckMultiplePIDs(recordingID)
--check if there are multiple Video PIDs in the file
tell application "EyeTV.app"
set input_text to my read_from_file((path to "logs" as string) & "ETVComskip" & ":" & recordingID & "_comskip.log")
if (count of (input_text as string)) > 0 then
set logdata to every paragraph of input_text
set logdata_lastrow to (item ((count of logdata) - 1) of logdata) as string
if (items 1 thru 19 of logdata_lastrow) as string = "Video PID not found" then
--multiple Video PIDs, rerun MarkCommercials until successful
set recrdingIDInteger to recordingID as integer
set rec to «class cRec» id recrdingIDInteger
set LogMsg to "RecordingDone found multiple PIDs for recording ID: " & recordingID & ", Channel " & («class Chnm» of rec) & " - " & («class Titl» of rec)
set PIDs to (items 44 thru ((count of logdata_lastrow) - 2) of logdata_lastrow) as string
set delims to AppleScript's text item delimiters
set AppleScript's text item delimiters to ", "
set PID_List to {}
set PID_List to every word of PIDs
set AppleScript's text item delimiters to delims
repeat with pid in PID_List
my launchComSkip(recordingID, pid)
repeat while (my mcIsRunning())
delay 5
end repeat
end repeat
end if
end if
end tell
end CheckMultiplePIDs
on read_from_file(target_file)
--return the contents of the given file
set fileRef to (open for access (target_file))
set txt to (read fileRef for (get eof fileRef) as «class utf8»)
close access fileRef
return txt
end read_from_file
on write_to_file(this_data, target_file, append_data)
try
set the target_file to the target_file as string
set the open_target_file to open for access file target_file with write permission
if append_data is false then set eof of the open_target_file to 0
write this_data to the open_target_file starting at eof
close access the open_target_file
return true
on error
try
close access file target_file
end try
return false
end try
end write_to_file
on launchComSkip(recID, pid)
if pid = "" then
set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --force --log " & recID & " &> /dev/null &"
else
set cmd to "export DISPLAY=:0.0; /usr/bin/nice -n 5 '/Library/Application Support/ETVComskip/MarkCommercials.app/Contents/MacOS/MarkCommercials' --force --log " & recID & " --pid=" & pid & " &> /dev/null &"
end if
do shell script cmd
end launchComSkip
on mcIsRunning()
set processPaths to do shell script "ps -xww | awk -F/ 'NF >2' | awk -F/ '{print $NF}' | awk -F '-' '{print $1}' "
return (processPaths contains "MarkCommercials")
end mcIsRunning
Mac mini Server (Mid 2010), Mac OS X (10.7.4), Lion Server, EyeTV HD, Turbo.264 HD
- This is awesome! I've been an EyeTV/aTV user for 3 years and lost the ability to edit out the commercials automagically. I got it marking them again on my most recent install, but would have to the compress in EyeTV and export to Apple tv manually to get rid of them.How can I contact you more directly for help on this? I am happy to make a donation to commskipper, and I'd make an in kind gift of some iTunes credit to you for assistance!Thanks!
- Jul 10, 2012 5:35 AM (in response to Brandon Riffel)I got it marking them again on my most recent install, but would have to the compress in EyeTV and export to Apple tv manually to get rid of them.Unfortunately, I don't believe this is possible yet with EyeTV products -- they have six different triggered AppleScripts, none of which allow running comskip before transcoding starts -- there's only ExportDone, but no ExportStarted, which would allow you to run comskip beforehand.This may be a (very thin) silver lining because if you mark commercials before transcoding, EyeTV/Turbo.264 reads comskip's .edl file (that has the start/stop times of all commercials it found) and clips out all marked regions, including any false positives from comskip. FPs happen. I've run it this way in the past with lores iPhone exports (that required absolutely no transcoding at all) and it sure is nice and saves a good amount of disk space, but you have to go back to the original if you ever have a false positive at an important part.It would also be nice if iOS/OSX/Apple TV respected .edl files for video playpack, thus allowing iOS/Apple TV commercial skipping on the device if you also provide the .edl, but I don't believe that this is supported at all.The only options I see are to request that Elgato support an ExportStarted triggered script, or to request that Apple support .edl video playback on iOS/Apple TV.
Subscribe to:
Posts (Atom)