May 2020 Summary

HIGHLIGHTS:

• Migrated KitCAT and Lib-Ray into Eclipse workspace / Git
• Started a TACTIC project for setup testing
• Purchased new monitor headphones and 24″ color-check monitor
• Reviewed sockets IPC programming in Python for KitCAT

May 18, 2020 at 4:00 PM

S-Type Icons for TACTIC/KitCAT

This last week, I started working on setting up a test multimedia project in TACTIC, and the first order of business is to define “S-Types” — “Searchable Types” for the project. I’m still a little fuzzy on the best way to create schemas for TACTIC, but as I understand it, S-Types are a fundamental part of the design, and they represent any kind of “object” you want to be able to keep track of in TACTIC.

TACTIC is also a very visually-oriented program, so it gives you the chance to identify these types with an icon image.

This got me thinking about what to keep track of in the program, how to break things down conceptually, and what sort of icons to use.

Fortunately, there are good icon sets on my Ubuntu Linux system that are free to use as graphical elements for new icons, and most of the old Open Clipart database has been rescued by the FreeSVG Clipart site, after the site went offline last year (there is something there again, now, but the FreeSVG site seems more functional at the moment). I also have pulled some elements from the icons for Blender 2.79 (such as the worldmap icon above). I think that’s particularly useful when the image represents the use of the object.

I realized that for the purposes of a multimedia project, the file-formats are much less important than the conceptual differences.

I’ve made over a 130 icons so far, some of them rather finely divided, but the lead image is a sample of 25 of the designs (left-to-right, top-to-bottom):

• “Audacity Project”
• “Ambient Sound” or “Soundscapes”
• “Sound Effects”
• “Voice Performance”
• “Recorded Music”
• “Photographic Image”
• “Painted Image”
• “Worldmap Image” (a spherical projection, as used for “sky” in Blender)
• “Papagayo” project file (.pgo)
• “Kdenlive” video project file
• Role – “Director”
• Role – “Actor”
• Role – “Character Modeler”
• Role – “Active Contributor”
• Role – “Passive Contributor”
• Blender – “Interior Set”
• Blender – “Character”
• Blender – “Extras Animation”
• Blender – “Mech Animation”
• Blender – “Compositing”
• Video – “Shot”
• Video – “Block” (of shots) or “Sub-Sequence”
• Video – “Sequence”
• Video – “Episode”
• Video – “Series” or “Season”

Of course, the goal is to manage an animated series.

Presumably, I will also be able to use matching icons in KitCAT.

I did create variations with the filetypes specified, though I’m not even sure if these will be useful in TACTIC. These variations account for many of the icons I made, so the number may be a little deceptive — for example, here’s the “FLAC” file format version of the “Voice Performance” icon:

It’s still quite a bit of design work. I don’t know for sure that this is a good investment of my development time, but it’s kind of fun, and I think the icons will improve usability.

I also spent some time getting KitCAT and also Lib-Ray imported into Eclipse to do some programming work on them. The KitCAT package actually runs and displays the mock-up (the same mock-up I’ve posted in the past).

May 23, 2020 at 4:01 PM

Why Go from Resource Space to TACTIC?

This was originally a reply to a comment asking, essentially, “what was wrong with Resource Space”? Why should we switch?

First of all, I want to say that Resource Space is an excellent DAMS (“Digital Asset Management System”).  And if all we needed were a DAMS, I would be very happy with it and continue to use it, especially since I’ve already invested the time to learn how to use it and maintain it.

I am particularly attached to Resource Space’s search and retrieval web interface. It’s very well-designed:

However, in our project, the DAMS is just one part of our system, so things get more complicated.

TACTIC is designed to be a whole-studio management system. While it’s true that the centerpiece is a DAMS, and the “asset” model is key to the way the program is organized, it adds a lot of things that we are currently having to manage with other software — these can broadly be described as “Version Control System” (VCS) and “Project Management” (PM) roles.

So the question is, then: what have we been using for those roles prior to now?

Essentially, our previous system (active in 2013-2019), looked something like this:

The green part (lower left) is client software on artists’ computers, while the blue part (upper right) is software installed our project server.

For individual assets, we’d do the work in our chosen open-source authoring software (the main ones are shown here, but there are others), and then saved to files on disk (in whatever operating system the artist is using — I’m on Ubuntu Studio Linux, but most of the artists were using Windows, and some might’ve used Apple Macintosh — for the most part, I didn’t even need to know this).

We used RapidSVN, specifically because of its cross-platform support on these three platforms (I actually liked kdesvn for Linux slightly better, because of the color-coding system, but RapidSVN is good — and I think it does sync faster).

In any case, we’d use the SVN client to synchronize files with the project’s Subversion version control system.

A number of people have criticized my choice to continue using Subversion on this project, and it does indeed have a lot of problems. However, for the most part, the problems it has are NOT the problems addressed by moving to a more-modern VCS, such as Git.  Since it’s always a hassle to change systems, I’m not that interested in changing to one simply because it’s more stylish, when I know it will have many of the same problems.

The real problem with Subversion (or Git, Bazaar, or Mercurial) is the conceptual misfit to our work. The assumption that the VCS is handling program source code causes a number of annoying management and performance issues. I’ll just list a few of the problems here:

• Since program-oriented VCS assume you are working on a software package, it forces us to break our project into rigidly-defined categories of “packages”, “source code”, “reference materials”, “documentation”, etc.  This is a poor fit for the loose accumulation of everything from concept art to scripts to Blender animation files that are part of our process.This forces us to choose between a rigid-definition of “source code” that may exclude too much and a broader definition that may produce enormously bloated “packages”.
• VCS packages also tend to assume that source code is small. Therefore it’s not a really big problem to move the entire packages over the wire in order to make comparisons or keep an entire extra copy on disk. This makes it a really bad idea to include heavy files, such as renderings or live footage in the collection of managed files. So, we have to start considering what to manage in the system, versus what to manage outside of the system.
• Although we have tried to maintain a distinction between our “source files” and “reference files” and a distinction between “episode files” and “library files” (and I think it is a good practice), the fact is that during the creative process, we might decide we need to move an asset into the library or make a reference file part of our source. These must be very intentional acts with existing VCS.
• VCS packages also assume that “compiling” — which we usually translate as “rendering” — is a fairly lightweight task that can be done on the client’s computer. Therefore, keeping rendered files is not a priority, as long as you can theoretically generate them. I know that compile-times can be quite onerous for large software packages, but render times on animation can take days on massively-parallel arrays of computers. There’s often an order of magnitude difference. The point is that rendering, unlike compiling, is an expensive operation and represents a project investment in an asset that should not be dismissed by the system.
• VCS also assume they are working with text files, and even when they have design accommodations to handle binary files, they can’t really do it intelligently. I’ve written some about how that might be improved, but it’s beyond the current technology. So this means that there are some operations that existing VCS workflow takes for granted (e.g. merging and patching source files) that we simply can’t do. In terms of VCS technology, this knocks us back to what was current tech in 1975. It’s a far more primitive system.
• And that means that most of the sophisticated features of modern VCS that programmers are interested in are almost useless to us. It really doesn’t matter which 3-way automatic merge algorithm your VCS supports, because none of them (yet) work on Blender files.

Of course, Subversion+Trac can also be thought of as a DAMS, as it does store our assets. Its interface as a DAMS leaves a lot to be desired, though:

You’ll note that no preview is available for this file. Here’s the same model, as viewed in Resource Space:

This is a much better preview! Resource Space actually renders the default camera from the Blender file, which is probably the right thing to do. But on the other hand, the Resource Space view gives us no version tracking information!

What is harder to show you is that the only reason this file is in both our Resource Space DAMS and our Subversion VCS is because I manually uploaded the file to each system. I am the only way the files are coordinated. And of course, this means there is nothing keeping them in sync, except me checking them. That’s laborious and prone to error.

What we want is for the versioning and asset searching to be the same system.

Lastly, there is “Project Management”.

What have used for that? A hodge-podge of all kinds of different applications for creating management documents and email, and also social-media chats and other communications channels.

In 2015, when we had two interns working on the project, we used an installation of PHP-BB to handle project communications and notes more formally. This worked to a degree, but was not a very good fit, as it was a little unclear how to use the thread system to match up to the work we were doing — we wound up using one thread for almost all of the communications.

In other words, we essentially have NO project management system. Theoretically, we could use a “ticket system”, which Trac includes, but we’ve never actually used it. The UI is probably a little too daunting for artists, and I’ve never seriously tried to push it, instead just handling things myself through emails.

All of these approaches also meant that every single communication had to go through me. There was really no way to break up into smaller teams, and this put a severe limit on how large the project could get. Managing a couple of interns was about my limit, and it cut into how much of my own work I could do.

How to Solve This?

There are, of course, multiple ways to tackle this problem.

We could try to create a better bridge between Subversion and Resource Space — perhaps use Resource Space’s ability to be configured to index a remote repository of files, and have Subversion be in charge of managing the files. It might then be desirable to write an extension for Resource Space to collect the SVN information on the file.

More recent releases of Resource Space DO have some version control, and we could try to adopt that system instead of Subversion.

We could try to find a better ticket system and adapt it to our use.

Any system of discussion also involved indexing the discussion to the particular subject — usually an “asset”  in the project — related to the project. Just doing this indexing correctly can be tricky in email:

Director:  “This is about the animation of the skipping in the latest version of the character animation for the train station exterior shot — S1E01-TB-2-B”.

Animator:  “Oh wait, is that the one I just checked in, or the one from last week? Because I made some changes.”

Director: “Oh. Wait. Let me look…”

And so on.

Or we could try to adopt a new combined DAMS/VCS/PM system that integrates all of these tasks into one system. That’s where TACTIC comes in.

Adopting TACTIC

In the TACTIC system, everything is based around “Searchable Types” or “S-Types”, which are generally the same as project “assets”, although they can represent other things. You choose which ones you’ll keep track of in your project setup.

So what happens is that version control, asset management, and project management, including these kinds of conversations are all automatically integrated. The animation sequence is a Blender file, which is one of my S-Types. It’s also a “Shot” which is another S-Type, and so the discussion will be indexed to both. As a director, I’ll make notes on the “Shot” or on the “Shot File”, and these will be then retrieved by TACTIC when the animator looks for the notes on what they are working on.

There’s also notifications and triggers, which allow us to start the conversation when this happens, not unlike receiving notifications from a new email.

Now, I’m planning to develop a context-aware client-side tool — “KitCAT” —  to make all this a little easier, although it can all be done through the web interface for TACTIC. That may take awhile, but the goal is a process more like this:

In this system, when the artist has the KitCAT plugin installed in their application (Blender, Krtia, Ardour, etc), then when they open up the asset file, they’ll have a button or other UI method to check for notes and make sure their file is up-to-date. This will then use inter-process communication (IPC) to talk to the KitCAT client (or launch it, if it’s not running), and then it will collect information about the artists’ version of the file, talk to the TACTIC server to look for any information about that asset there, and then bring it altogether into a single interface.

Most artists only deal with one or two authoring applications, and changing applications “breaks their flow”. So it’s better if the client is accessible from the application, and if they don’t have to go to open a web browser to contact the server. This way, the program can collect that information in the background and then pop up a window with the collected context, including any new notes, whether the file needs to be updated or committed, and so on.

In order for TACTIC to do all this, though, it has to be the DAMS the project is using internally.

The biggest failing of TACTIC for our purposes is that it lacks any special knowledge of open source production authoring software: Blender, Ardour, Audacity, Inkscape, Krita, Gimp, etc. So it can only understand those things as “files”.

That’s because TACTIC was originally developed in a studio that used Maya and other industry proprietary software.

What that actually means in terms of practical problems is not something I’ve been able to learn yet. But my guess is that it may not be able to automatically extract metadata from files as readily as, say, Resource Space does.

I decided that the best way to handle this would be to incorporate application-specific introspection programs to do this work in the various applications. This is clearly a good approach in Blender, as you can find out a lot about the currently-loaded Blender file exploring the ‘bpy.data‘ namespace. And it is possible to send that data via IPC to another program using Blender’s python interpreter (I know this works, because I tried it out early in the process).

Other programs will require additional work — Ardour, for example, uses Lua as a scripting language instead of Python. And it remains to be seen if its interpreter will allow for tricks like this.

However, if the plugin/IPC approach fails, we can still fall back to file-based analysis, where we run KitCAT directly on the file, and extract information from it. RenderChan includes application-specific file analysis, and we might use that directly, or just implement similar features.

Finally, even if we lack application-specific analysis for a certain program, we can always fall back to the same filesystem metadata that Subversion and other VCS programs rely on to determine currency with the files on the server.

Before KitCAT is developed, we’ll be limited to what TACTIC can do through the web interface and what it can determine about files on the server. Another approach to this problem would be to extend what the TACTIC server can learn about files on the server side. It may be possible to add this functionality via plugins, and it’s even possible that some plugins already exist by now for the program file formats I want (I have not seriously looked for that yet).

Will it work?

I don’t know. Bu that’s what my current experimentation with a local TACTIC server is meant to find out: will it work? Is it better than our existing system? Is this the right approach to establish our backbone studio software?

I got stopped on this for a very long time, and only this year started pursuing this in detail. So far, I’ve got a sample project and S-TYpes:

It looks very promising, but I have a lot to learn.

Katrina Niolet created a mock-up implementation of KitCAT which runs, although it doesn’t really do anything yet. I am considering proceeding by adding functionality to this mock-up:

Alternatively, there is another program called “TACTIC-Handler” which is just implemented as a Qt-based native client for controlling a TACTIC server via the web-API. My current thought is to use this as a guide for how to write such software, rather than working directly with that code. But I haven’t really tested it out, yet.

One problem that does come up is that TACTIC and TACTIC-Handler are under the EPL-1.0 (“Eclipse Public License”), while KitCAT is currently under GPL3 (“Gnu General Public License”). These licenses are not quite compatible, although both are considered “free” and “open source”. This would make it awkward to simply port modules back and forth between TACTIC-Handler and KitCAT.  I’m not sure what I want to do about that, as yet, but my current plan is just to re-implement what I need around the KitCAT frame and simply use Handler as a guide to working with the TACTIC web API.

We could conceivably change our license, but I’m not sure if it’s worth doing.

Comparison to Resource Space

Getting back to the decision to retire using Resource Space: should we keep maintaining Resource Space as, perhaps, an alternate way to access our assets? Or should we drop it, and instead simply expose TACTIC’s DAMS search interface to the web?

The truth is, at this point, I don’t know that TACTIC’s UI will be an improvement. It might. It will most likely have a more sophisticated and therefore more complicated search interface. This might be a little much for publishing our assets to the outside world.

On the other hand, we’d be directly exposing the interface we’ll be using within the project. That will make it a lot easier to maintain than a service that essentially amounts to “public relations” and that we might not be constantly checking up on. In other words, we’d have the same problems we currently have keeping Subversion and Resource Space synchronized — and the solutions for that would be similar, if we decided to do it.

But it’s simpler not to, and just drop Resource Space — or at least not use it for sharing our TACTIC resources.  And that’s really the only reason I’m considering dropping it.

I hope this clears up the reasoning behind this project, but also lets you see that there’s a lot I don’t know yet, and so haven’t made final decisions about. I know it’s taken me a long time to get around to doing this evaluation, even though I’ve been talking about it (literally) for years. I don’t really know what to say about that, except: “Hey, I’m finally doing it now.”

May 29, 2020 at 4:00 PM

“Lunatics!” Sneak Preview / Sales Tax on Patreon

Sneak Preview

I hope this isn’t too much in the way of spoilers, but I know we’ve been taking a long time, so I’m releasing a little animation to show you what we’ve completed already.

The video above shows about five minutes of fully-rendered (but not fully composited) animation for our pilot episode, focused on character action. We’ll be adding a few tweaks to it, but it is very close to what we’re trying to achieve. Otherwise, we’re primarily trying to get the rest of the episode (including almost all of the mechanical animation shots) up to the same quality.

These are probably the most challenging animation sequences, considering the coordination of characters and sound. There are still a few goofs in this — a few short pieces of animation in the “Press Conference” need to be reanimated. Sergei’s lips don’t move when he’s talking at one point (I think maybe the animation channel was turned off?) and there’s an interference problem with Reporter #1’s eyelid, that I’ll either need to re-render or carefully fix in compositing. And there’s no background music at all in the Press Conference scene, which is how it’s supposed to start out, but there will be some leading into the following sequence.

Sales Tax on Patreon

The immediate reason I’m sending out this message to all of our patrons today is to let you know about a change Patreon is rolling out June 1st: In order to comply with changes in tax law, they will now be computing sales tax on pledges which include taxable benefits, depending on your jurisdiction and the nature of the pledge.

For most of you, this will make no difference, because most of our tiers do not include physical products, and for most jurisdictions, taxes will be limited to those. A few will make digital download benefits taxable, and that may include the “streaming” levels on our project. I’ve reviewed our tiers to make sure they are tagged appropriately for tax calculation.

Furthermore, as I understand it, the taxes will simply reduce how much we will receive from your pledge, not increase the amount you pay, in as much as they apply at all.

Of course, as you know, we’ve NEVER actually charged for your support on “Lunatics!” Project, because we haven’t released our first episode yet. I know it’s been a very long time in coming, and I very much appreciate your patience. But part of the reason I didn’t want to make this project monthly was because I didn’t want to charge you in advance and then have that hanging over us. This way, there’s no charges until we produce a result.

This is also the first post I’ve sent out to many of you for a very long time, though, so I’d like to let you know more about our project status.

Film Freedom

We recently created a separate, monthly account, called  “Film Freedom” Project, which is dedicated just to supporting our infrastructure work.

Film Freedom Project

If that’s what really interests you, and you’d prefer a small monthly donation, then I invite you to support it. We haven’t really been promoting it very strongly yet, but we do have some posts up, including the first “Film Freedom Podcast”.

Our primary infrastructure project is to develop a full “Virtual Studio” around the open-source Southpaw “TACTIC” studio project management server, various open-source authoring applications (Blender, Audacity, Ardour, Inkscape, Krita, etc), and a context-aware client, to be called “KitCAT”, which will facilitate using TACTIC with these open-source packages.

KitCAT isn’t strictly essential to the virtual studio, since TACTIC can be managed entirely through the web interface, but we have a mock-up implementation, and I’m working on integration with Blender. I’m also testing project setup on a local TACTIC server, so as to write better documentation, create templates, and so on.

The second major infrastructure project is the “Lib-Ray” project, to create a distribution format for fixed-media, which is entirely open-source, as an alternative to the proprietary DVD and Blu-Ray formats. This derives from a Kickstarter-funded startup project, that unfortunately got bogged down with the development of a demonstration software player — a key component for testing the standard. Our initial design did not go smoothly, and we’ve had to restart with different libraries. I’m now currently assembling the existing source code and testing code so it can go up on GitHub — and also so I can finally get it working.

We’re also trying to put together more tutorials on using open-source production tools, as well as recommending good ones from other sources.

Lunatics! Progress

I have posted a LOT of updates for the $12 and $20 tiers on “Lunatics!” over the last several months. Almost all of these are now public, since I set them to go public after a week. As this has always been intended to be a “Open Movie” project, I don’t really believe in creating permanently exclusive content for it.

Unfortunately, there doesn’t seem to be a way to configure Patreon to notify you when that happens, and just making a public post doesn’t either. I’ve also gotten a couple of complaints about too many notifications when I was sending out posts to all tiers. So I hope I’m striking a better balance now.

Some of the highlights over the last few months, though, include rendering significant parts of the more demanding animation on a render server provided by Andrew Pam. We’ve also discovered that our studio monitor has very poor color calibration, and we’re working on a means to color-correct what we’ve rendered and also to acquire a monitor that will allow us to avoid this problem in the future.

About half of the episode (by screen time) is either complete or nearly complete, and the remaining half represents considerably less complex animation. I’m mostly working on organizing the mechanical shots — I really want them to look nice, and that’s partly why I’m hoping to get the color issues sorted out before finalizing them.

One troublesome sequence is the very beginning of the episode. These shots were animated early, but there have been some changes to the model and animation files, and getting everything properly merged so that they can be rendered correctly from the sources is tricky.

My last few posts have been about our infrastructure project with TACTIC. Probably I really should be posting those on the Film Freedom page, but there are so few people watching it yet. So I’m still posting them here.