Followers of this blog should remember me talking about the production asset management system back in june this year. Since then I tried to find a nice way to handle the files in our studio. I looked the on-the-shelf tools like AlienBrain, Shotgun etc. But I wanted to have a system which is structurally not that much complex, and as open as it gets so I could instantly add new features to it. And I think, I was attracted by the idea of being the writer of an asset manager system. So I decided to write of my own system.
Those days we were using a simple MEL script to let Maya to decide where to save the file by looking a template folder structure stored in the file server. With some limitations, this little script was doing its job very well.
In my first attempts to an Asset Management system, I had a lot of ideas about what an asset manager should do and a little bit of idea about what it shouldn't do. I tried to gather those ideas to form a system, but I couldn't manage it. Somehow I couldn't do the thing. I drew graphical charts, schemas explaining the data flow, the program flow and the general structure and I started coding of the Asset class in Python. But again somehow I couldn't manage to have something meaningful. So I let it go.
Because, I couldn't do anything about coding of the asset manager, I decided to rewrite the ProjectCreator tool by using Python. It was originally written in C#. We were using this tool to create empty projects, with a predefined folder structure and with definite number of shots in some specific folders and to add new shots folders inside that project structure. The tool was written very badly around the interface code by myself. And because of the design, we were not able to change the project structure independently for one project to another. So I decided to have a proper object oriented system. Which also lets me to script the regular things like adding folders to all of the projects and sequences at once.
I've started coding of the classes those I'll going to need, like the Database, Project and the Sequence classes. They were doing some basic things. But, the Sequence class was especially special for me, because, some how I saw the way to the asset manager in it. While coding the Sequence class, it is connected it self to the Asset class I had written before. So the basic problem was solved. I had all my basic structures to form a very little living asset manager system.
At the beginning, it was just saving the files to proper folders with proper names (same thing with the previous MEL based system). I had to code an interface for it to let the artist to use it. I decided to use PyQt. We instantly switched to the new system.
In time, the system evolved to a python module and I named the system as AssetManager with peace of mind. The system is evolving very quickly. It is already doing things like, finding all the assets those a user did for a specific sequence or shot, tracks the file versions in a very simple but solid manner (by saving every version with a different version number :) and not letting the user to save over an old version), keeping track of the referenced asset, and updates the references automatically, tracks shot data like shot timings, shot descriptions etc.. The best part of the system is, it is working under any platform that Python is supported. It means, it works under Maya, Nuke and Houdini. And I'm trying to find a way to add it to Photoshop (probably I'm going to write some Java code for it). And while adding all of those features, I always tried to keep the generality of the code.
By having a working system like that, now it is much easier to think about the new features that the system needs. So, I can see the next two steps in front of me.
All the artist are very happy to be able to use a system like that, and they are keen to see new features coming constantly.
Those days we were using a simple MEL script to let Maya to decide where to save the file by looking a template folder structure stored in the file server. With some limitations, this little script was doing its job very well.
In my first attempts to an Asset Management system, I had a lot of ideas about what an asset manager should do and a little bit of idea about what it shouldn't do. I tried to gather those ideas to form a system, but I couldn't manage it. Somehow I couldn't do the thing. I drew graphical charts, schemas explaining the data flow, the program flow and the general structure and I started coding of the Asset class in Python. But again somehow I couldn't manage to have something meaningful. So I let it go.
Because, I couldn't do anything about coding of the asset manager, I decided to rewrite the ProjectCreator tool by using Python. It was originally written in C#. We were using this tool to create empty projects, with a predefined folder structure and with definite number of shots in some specific folders and to add new shots folders inside that project structure. The tool was written very badly around the interface code by myself. And because of the design, we were not able to change the project structure independently for one project to another. So I decided to have a proper object oriented system. Which also lets me to script the regular things like adding folders to all of the projects and sequences at once.
I've started coding of the classes those I'll going to need, like the Database, Project and the Sequence classes. They were doing some basic things. But, the Sequence class was especially special for me, because, some how I saw the way to the asset manager in it. While coding the Sequence class, it is connected it self to the Asset class I had written before. So the basic problem was solved. I had all my basic structures to form a very little living asset manager system.
At the beginning, it was just saving the files to proper folders with proper names (same thing with the previous MEL based system). I had to code an interface for it to let the artist to use it. I decided to use PyQt. We instantly switched to the new system.
In time, the system evolved to a python module and I named the system as AssetManager with peace of mind. The system is evolving very quickly. It is already doing things like, finding all the assets those a user did for a specific sequence or shot, tracks the file versions in a very simple but solid manner (by saving every version with a different version number :) and not letting the user to save over an old version), keeping track of the referenced asset, and updates the references automatically, tracks shot data like shot timings, shot descriptions etc.. The best part of the system is, it is working under any platform that Python is supported. It means, it works under Maya, Nuke and Houdini. And I'm trying to find a way to add it to Photoshop (probably I'm going to write some Java code for it). And while adding all of those features, I always tried to keep the generality of the code.
By having a working system like that, now it is much easier to think about the new features that the system needs. So, I can see the next two steps in front of me.
All the artist are very happy to be able to use a system like that, and they are keen to see new features coming constantly.
Comments
It might be something I wouldn't mind working on.
About the addition of an SCM:
At the beginning of the design, I've really tried hard on thinking about the idea of adding an SCM system. Mainly about what is going to be its benefits to the system. I knew that they can keep track of the changes on files, keep track of different branches of changes, roll back to a previous version of the project, can merge changes from different branches etc.
But non of them were needed for our pipeline. Nobody is working on the same asset so we don't need any branching, if we need to roll back to a previous version, then we can do that easily with our regular files. Again, we can easily find the person responsible from a version of an asset with our system. And SCM systems need a local copy of the project on the users machine, which in our case is not feasible cause our project sizes ranges from 50 to 250 GB. And most of the SCM systems around (Git, SVN, Mercurial etc.) are written for text based code, although they support binary files, they don't recommend to keep track of changes of files bigger than 10 MB or around ( at least Mercurial doesn't ). And Maya's ascii file format is not something you can track the changes from one version to another. It's a complicated format, it is not like a group of code blocks etc. (although it is basically MEL)
So, I decided not to use an SCM and keep all the new versions of an asset as a complete individual file right beside the previous versions. I've just increased a version number inside the auto-generated file name which has all the basic information about that file ( it has just the basic info so it is not enough for a real production asset manager).
I hope it comes meaningful, to not to add an SCM under these circumstances.
About the sharing of the code:
I disabled the Google site, because it was meaningless while I was developing it by my self. But I'm going share it later on, when my current project finishes.
I was searching through all the ProdAM and DAMs available, and did not find somthing really useful free / openSource for Production use.
hope your project is finished soon, I am eager to get this into my distro (called openArtist).
Shotgun is an closed and paid service/system. But, as you pointed, it is a very powerful one.
I've started to design a system which will be an open source alternative to Shotgun. But it is to early to introduce the system for now. It will be written with and on Python/SQLAlchemy/Elixir/PyQt/SQLite (or PostgreSQL the database will be a choice of the user actually). Be excited!
I'm going to open a codes-google account for the system (just waiting SourceForge for an answer for a project name takeover issue). Then I will call everybody to participate...
I've already designed the database schema, and I need some help with that...
Once it's up I will contribute my thoughts on this matter.
Cheers!