Ken,

These are great feature requests.  I'm copying the dev mailing lists in 
the Helix DNA Producer project for feedback and to look for volunteers 
interested in making the changes based on the specification we 
eventually come up with.  In fact, if you are interested, I'd like to 
explore the option of you making the changes yourself.

Of course, if you choose, you could simply download Helix DNA Producer 
source code yourself, under the RCSL (commercial license - royalty free 
for Helix Producer) and make the changes you need for your purposes. 
 Ideally, any changes made would be done under the RPSL (public source 
license) so that changes go back into the community and others can 
leverage from them or even build on them.

I see these ideas as being divided into two parts, both related to some 
form of cross-platform interprocess communication:

   1. Communication from the producer app to other apps to report
      status, most importantly,  the process state (zombie/active).
   2. Communication from other apps to the producer app to change
      properties dynamically during an encoding session.

I'd like to discuss these separately since one is easy and the other 
quite hard but first some more background:

The PID file (-pid option on the command line) is a means by which this 
communication is proposed to occur.  This makes good sense, as pointed 
out below, for the following reasons:

   1. Most Platform Neutral - Only requires read/write access which is
      relatively simple/similar across OSes.
   2. Most API friendly - Anything from a scripting language to C++
      application can interact with a text file
   3. Safe - Solution is not vulnerable to parent-process crashes/errors.

The properties you propose can be broken into two general categories as 
suggested by the means of communication above:

   1. Read-only properties (messages from producer)
          * pid=1024 
          * timestamp=10123456789
   2. Read/write properties (messages to producer)
          * run=true 
          * archive=false 
          * title=World News Tonight with Peter Jennings

I interpreted the later 3 as properties you would like to set on an 
encoding session.  Please correct me if I am mistaken.

1.  Extensions to the PID file for reporting status of the encoding process

The most important idea here is the heartbeat..  That is, writing a 
timestamp to the PID file on a regular (and probably configurable) 
basis.  This is a fantastic idea and would prevent the need for having 
code that checks status of a process through OS-specific methods.

This provides a simple interprocess communication channel from Helix 
Producer to other applications.  The question is how far do we want to 
go with this idea.

One option is to leave the pid file format the way it is now to avoid 
breaking any code out there that assumes a pid file has a single PID 
only.  The timestamp on the file can still easily be updated so that 
applications can check to make sure producer is still active.

The other option is to change the file format and allow more than just 
PID in that file.  Then we could add other properties in addition to 
timestamp and PID such as some of the info that is printed on the 
command line status including:

    * status (initializing, encoding, merging)
    * encoding duration or %
    * total duration

The problem with the later is the possibility of breaking any apps that 
already depend on the PID file.  There is also a concern about "feature 
creep" where folks start to look at this as a means of getting status 
from Helix DNA Producer.   However, as explained below, RealNetworks has 
specific vision of how to implement  status on the command line which 
would be far more supportable in terms of adding new functionality in 
the long run.

If we do go this route, my vote would be for a simple (not XML) syntax. 
 This is because the most likely use case is access from script language 
and the need for a highly developed XML structure really does not exist. 
 However, it would be useful to hear back from folks on which format 
they prefer. That is:
Simple Format:
pid=342
timestamp=3454534243
or XML format:
<session>
   <pid>342</pid>
   <timestamp>343435435</timestamp>
</session>

I think the first format is much easier to parse and work with.

2. Extensions to the PID file for sending commands to Helix Producer

While this is interesting, there are two challenges to this.
1. Helix Producer does not really support dynamically settable 
properties as suggested below.  

For example, one cannot trigger archiving on the server, either from 
producer or even from the server (archiving can only be scheduled to be 
on or off always and a roll time/size can be defined before the stream 
starts).  

Nor can you set the encoder in pause mode, though this has been 
requested.  This is a useful feature but would take a significant amount 
of engineering work and is something that would best be done by the core 
helix development team.

There are actually very few properties of an encoding session that can 
be changed on the fly during an encoding session though this is 
something we have considered changing in specific cases.

Also, the mechanism for remotely controlling encoding sessions is 
something that we feel should be solved in a more general extensible 
manor.  The current thinking is to build an XML/RPC based solution that 
would enable Helix DNA Producer to listen for commands over an HTTP 
connection.  If we go this route, we very likely would consider adding 
command line options to submit XML requests via command line instead of 
an HTTP socket to enable scripting applications to perform the same stuff.

An XML/RPC based control mechanism is something we are thinking of in 
terms of our roadmap but as of yet have not firmly decided to commit to 
this feature publicly.

Cheers!
Steve McMillen

Ken Martinek wrote:

> Hi Steve, I'm Ken Martinek at Ascent Media (fomerly Liberty Livewire 
> Corp).
>
> You may remember my previous correspondence relating to various ideas 
> for improvements to the cmd line producer.
>
> Previously I'd wrote you asking your opinion on changing producer to 
> write some form of datetime indicator to a file for purposes of an 
> external 'heartbeat' monitor application. I'd like to extend that idea 
> a bit... what if you had producer do that very same thing to the PID 
> file every minute or so --not only doing a setModifDate() on the file, 
> but perhaps also writing the date into the file itself (as a UNIX time 
> integer)? This would allow an external piece of software to determine 
> if the process is still active, perhaps with lower overhead than than 
> spawning a platform dependent command to see if the process is running 
> (ps L/UNIX, tlist Windows).
>
> But the main benifit of this concept (my biggest need right now) would 
> be the following -- not only could you write a time stamp to the file, 
> but it would become a (more platform nuetral) method of interprocess 
> communication. For example, we could write a String/int to an 
> additional line in the PID file which producer reads at some interval, 
> which might be labeled 'run' or 'pause'. So if I have forked the 
> producer --and no longer have an input stream to send Ctrl-c, I could 
> issue the equiv of a TERM signal via that line in the PID file 
> ("run=false").
>
> Suppose that the PID file contained other attribs, like 'archive' 
> ---while producer was already running, it could send a flag to the 
> server that indicates the straem should be archived --without 
> interupting the stream that a live audience is viewing --or perhaps 
> another param to dynamically change some characteristic of the video 
> encoding. Maybe the steam title?
>
> # producer PID file
> pid=1024
> time=10123456789
> run=true
> archive=false
> title=World News Tonight with Peter Jennings
>
> In the above sample some params would be read-only to outside apps by 
> virtue of the fact that producer owns and will always write to them 
> (pid, time). The others could be parsed as runtime args whose names 
> match existing (or new) command line args. Obviously the better 
> approach would be an XML based format.
>
> Though there may be other/better ways to achive these same benifits, I 
> think this one has a few benifts that are really important :
>
> -Dynamic - one could change the runtime characteristics of the stream 
> without have to BOTH maintain AND read write to an input/output stream 
> to the Process (if the process that spawned the producer crashes, the 
> IO stream is lost until producer is restarted)
> -Most platform neutral - one only needs to get handles and read/write 
> to the files, which is relatively simple/similar across OSs (and 
> identical in Java of course), and not vulnerable to parent-process 
> crashes/errors.
> -Most API friendly anything from a text editor to a leightweight 
> homebrew app can interact with a text file.
>
>
> Please let me know what you think. Aso, I'd be happy to alpha test any 
> other software you may have that achieves the same end benifits.
>
> Thank You,
>
> ken
>
>