Player in Clutter and DBUS

Notes on the digital signage player built with Clutter and DBUS: metainformation, roadmap, user manual and test suite.

Translation of the original in Spanish. Read it in: CatalàCastellano

Part 6 of 7Clutter notes
  1. Clutter concepts
  2. Notes on Clutter
  3. Installing Clutter with jhbuild
  4. Clutter snippets: D-Bus, GStreamer and ClutterScript
  5. Some notes on Clutter development
  6. Player in Clutter and DBUS
  7. Player in Clutter: continuous integration

This page was on the oriolrius.cat wiki (2009–2012). I bring it back here as it was.

It is a player that feeds on clutterscripts to display scenes, the playlist is received via DBUS and you can interact with the player through this interface.

Resources

Cookbook (short user manual)

./player.py --help
./client.py --help

And abusing a few lines, let me expand a bit more. The player is player.py. It can be launched without arguments, in which case it starts and stays idle, or you can pass pages as arguments, e.g.:

./player.py page1.json page2.json page3.json [...]

In this case it is as if we did 3 'load', 1 for each page, and a 'next', that is, we load the pages and launch the first one.

The GUI also has a few commands:

  • 'q' to quit
  • 'n' runs the 'next' command
  • 's' runs the 'stop' command
  • space for 'start' and 'pause'

Besides that, client.py allows managing the player's dbus interface. It has all the commands, it works in one of the following ways:

./client.py -c start
./client.py -c stop
./client.py -c pause
./client.py -c load page.json
./client.py -c next
./client.py -c clear

Metainformation

Below we define the elements a digital signage channel is made of, after all the content a player has to play.

"Channels are made up of Pages which in turn allow splitting the usable area of the screen into Zones. Each of the Zones must have at least one Basket associated with it."

Canal -> Páginas -> Zonas -> Cestas -> Media

Descriptions:

ClutterScript

A ClutterScript could be a page of the channel, then a channel would be made up of several ClutterScripts each of which represents a page of the channel.

Since our system is the initial version, we only support pages with one zone or two zones. In the following MCP screenshot you can see several templates; we will initially only support template 1 (the selected one) and 4 (video zone with a footer which is the ticker zone)

Page templates of a channel
Page templates of a channel

Inside a zone (never mind if it is the big zone or the small one/ticket) we can have a basket (playlist) assigned; this playlist will contain a list of media to show: video, image or text ticker.

Therefore our clutterscript that defines a page will look similar to:

{
  "My Scene" : {
    "id" : "pagina1",
    "type" : "ClutterStage",
    "title" : "ClutterScript test",
    "color" : "white",
    "children" : [
      {
        "id" : "zona1",
        "type" : "Zona",
        "x" : 0, "y" : 0, "width" : 1024, "height" : 768
        "children" : [
          {
            "id": "cesta1",
            "type": "playlist",
            "repeticiones":"0", (0:ninguna,-1:loop,n: num de veces)
            "children" : [
              {
                "id": "item1",
                "type":"video",
                "file":"/path/filname.avi",
                "sound_volumen":"0",
                "scale":"yes"
              }
            ]
            "children" : [
              {
                "id": "item2",
                "type":"image",
                "file":"/path/filname.png",
                "time":"0",
                "scale":"yes"
              }
            ]
          }
        ]
      }
    ]
  }
}

How a player works

It has a FIFO queue of stages/pages that it plays sequentially. In turn each of the pages has the elements commented above, on which you cannot interact beyond setting play/pause/stop.

When the timeline of a page/stage runs out the next page is loaded and shown. Doing POP on the FIFO queue. So an element of the FIFO queue that is being played is no longer in the queue.

If a page is in infinite LOOP, this page is taken out of the FIFO and never allows it to move to the next element of the queue, until some external element sends it a 'next'. This forces the next element to be loaded, because otherwise on its own it will never leave the loop.

When a 'load' is done what it does is add an element to the FIFO queue that will be played when appropriate.

A 'clear' will only empty the FIFO queue, it does not affect the element in play.

Player-Clutter: Roadmap

version 0.1 ~end of January

  1. Playlist based on ClutterScripts. Estimated deadline: 4 and 11 December for the first 2 points Done
    1. First of all, we need to finish defining what the structure of a playlist is. Will it be the same as in, e.g., Signtech? This structure was: Done
      1. Sequence of videos and/or images. In the case of images, a field for the exposure duration of the image was indicated. Done
      2. Possibility of ticker. Independent of the previous sequence, it simply contains a text that scrolls from left to right the whole time. Done
        1. Doubts: Regarding the ticker, 2 doubts come up. First of all, the signtech-type ticker had no time control or number of repetitions, since it acted as a slave. The ticker is currently implemented like that. However, I understand that this should not be the case, but rather a duration time or a number of repetitions should be set, right? What would be the preferred way? On the other hand, currently the duration of one pass of the text in the ticker is constant (and hardcoded). This means that if the text is bigger it will go slower and if there is more text it will go faster... should this be configurable? based on a constant time or a constant speed?
          1. ANSWER: the ticker behaviour must be:
            1. the ticker is shown at a speed of X pixels/second (parameter defined in JSON) Done
            2. a maximum time for which this ticker must be shown may or may not be defined, for example, if I set 1min the ticker will be shown in loop for 1 minute and then it will stop being shown Done
            3. loops=X, if X=n it is shown n times for n>0, X=0 it is shown an infinite number of times Done
            4. if all the logic gets very complicated, implement only the speed one and something is something... the normal thing is for a ticker to be a slave as happened in signtech, the rest of the functions are in case the ticker zone were the master as you will have imagined. In the end I implemented the complete logic.
    2. With the structure of the playlist defined, we have to think about how to map it in the json. What is clear is that in the json we will directly have 3 objects: Video, Image and Ticker to which we can map the properties; what I am still not clear about but I don't think will be problematic is the way to define a sequence. @oriol: I don't know if you have thought about this, if not, don't worry, it's just that I haven't had time. Done except for the properties to be added according to the previous point in case of need. complete.json is a sample page.
    3. Once the previous point is resolved, it is enough to do, if necessary, some glue to implement the sequences, show the scene with the different elements, etc. However, this point would be carried out once we have the 3 classes working (at least prototyped). Estimated deadline: 1 January to set it up, 8 January to test and finish squaring all of point 1 and all of point 2. Done
  2. Supported actor types. Each type of element will have its own class, each one with the specific properties we need: Estimated deadline: 19 and 26 December for this point
    1. Video, via gst. It will inherit from ClutterGstVideoTexture. Done, although there is a problem with scaling. Currently it scales automatically to the screen size, since I have not found a way to get the width/height of the video until it is running. The doubt also comes up whether when scaling the width/height ratio must be kept constant or not.
      • ANSWER: leave it like that although it will have to be supported: scaling keeping x,y ratios
    2. Image. It will inherit from ClutterTexture and will have a duration property (if it is not set through a timeline) Done, although the width/height ratio doubt remains here too.
    3. Ticker. It will have the text to show and, in the future, it will be able to extend its features (text size, typeface, colour, etc.) Done except for determining the speed, already commented above.
  3. dbus interface. Once we have the actors and the supported playlists, this interface will have to be created. Below I try to define a bit more and throw out some idea of how to do it: Estimated deadline: 15 and 22 January to implement it, 29 January to test it, and solve problems
    1. play, pause, stop: these 3 methods will allow starting, pausing and stopping the player. In fact, they will act directly on the stage, and it will automagically work for all the actors (if it doesn't quite work out of the box, we will have to do the glue so that it works as I say, but it may well be automagic, according to the clutter docs). Done
    2. next in pl: it will finish the 'timeline' of the current image or video and move on to the next one
      • downloads current stage and moves on to the next one Done
    3. clear pl: the behaviour has to be defined: would it stop the player? or would it go on until the end of the pl but not start over? that is, would it destroy the whole stage?
      • deletes all the stages we have in the cache and nothing else, if the user wanted a pause/stop they should have launched it Done
    4. load pl: a json could be passed as a parameter I understand... in this case... would it be like a 'reset' of the player? or would it go on with the current pl until its end and afterwards load the new one?
      • it would be similar to an append stage to my list of stages, do you follow me? Done
    5. I have created the dbus interface and a client: client.py that allows launching commands via dbus to the player Done
  4. Fullscreen
  5. Integrate in moblin Estimated deadline: the previous point and this one, we could set 1 week, 5 February, to try to square everything, I don't quite know how problematic the integration will be... the ideal thing would be to test while it is being developed. If you pass me a virtualbox image or similar (or a link or some indication) I will set it up and thus avoid surprises at the end

Design issues to be resolved:

  • fullscreen: it seems that fullscreen behaviour is very dependent on the graphics card and the driver. It also depends on the window manager. I have not managed to put the snippets in fullscreen with my card/drivers/manager. I created a snippet in C to check that it was not a problem in the python bindings (I also tried with the program that supposedly worked) and it has not been possible either. I have seen that there are quite a lot of people with this problem, and that depending on the order of the initialization of the main stage the behaviour may differ, so I think I know how to put an application in fullscreen (if I managed to make fullscreen work for me)
  • dbus interface: I have not detected any particular problem in making the dbus interface.
  • supported actor types: I have tested mainly with video (via gstreamer) and I don't see that there will be any bigger problem. Images are supported without problems and the ticker is no problem with an animation.
  • playlist based on clutterscript: for this section I have been considering how it could be done. My initial intention was to inherit our elements (video and ticker) from a Clutter Stage or Actor and call them directly (I created the complete.py snippet with basic.json to go in that direction). I thought ClutterScript would be able to interpret all the objects of the Clutter hierarchy, but that is not the case, its knowledge seems to be limited to the objects defined in the library. If that were the case we would have to check the behaviour of the functions that can be called from clutterscript (with the type-func element) to create those objects. After Friday's conversation, I started investigating how to implement the parser, and how to make it call our objects. Investigating this matter I went over the glib object system, gobject, and reading the docs I saw what was failing: for ClutterScript to be able to load our objects that follow the ClutterActor hierarchy, what was missing was registering these classes as gobject types. Once this was resolved, I managed to load our classes with ClutterScript. Once this question was resolved, two new questions came up:
    • How to get our video object to be an object within the ClutterActor hierarchy, since in all the snippets we have, the 'players' make use of a texture, but they are not objects of the texture type. To understand this well I did have to resort to the gstreamer documentation, since the operation of a gstreamer player has a somewhat complex organization, which is difficult to understand without knowing the gstreamer architecture. Once the operation was understood, I now have a Video class that inherits directly from ClutterGstVideoTexture, so we are inside the Clutter object hierarchy. This point, together with the previous one, are resolved in commit 73
    • How to 'adapt' traditional playlists (if I remember correctly, a series of videos and images that were shown in an ordered way could be defined, and also the possibility of a ticker, independent of the sequence of videos and images, at least that was the case in signtech, I don't remember if it was the same with the other manufacturer). I think this will not be a problem to solve, but in the roadmap I put it in first place. We would indeed have to define the playlist format we should support.

version 0.2

  • plug-ins
  • complex scenes: multiple zones, multiple sizes, check behaviour of master zones.
  • actors
    • webkit
    • flash (from webkit?)

version 0.3

  • plug-ins
    • they capture signals from the screens
    • they connect playlists
    • interactivity
    • etc
  • widgets = clutterscripts + plug-ins

Player-Clutter: Milestones

Test suite

Tests that should be done on the player to know whether it is already stable enough to make the release:

Playlist

The basquest/playlists are likely to consume all the memory of the machine if they are too big. So it is proposed to generate playlists of up to 1,000 items combining videos and images to see what the behaviour is.

For the moment this has to be validated with a stage with a single zone.

Stage

  1. Create a stage with several zones (2, 3, 4 and 5) with slave playlists and one master that can be rotating, check that the behaviour is correct.
  2. Try with a stage of two zones (master and ticker) putting a giant playlist in the master zone and check that the system stays stable while the ticker keeps going round infinitely.