Sort Files Into Folders Based on Prefix

[graham728]

Hi,

Wondering how I can create a watch folder that reads the first 6 characters of a file name - it then looks for a folder that starts with the same 6 characters in another folder and if a match is found it will drop the file into that folder with the option to place within a subfolder.

We use tracker numbers for all our projects. The project folders start with this number and source files/projects/visuals etc start with the same number.

For Example:

Source File:
CP1234_After_Effects_Project_File.aep

Sort into folder that begins with the same tracking number CP1234_

Root Folder >>> CP1234_Campaign_Edit_Name
Subfolder >>> After Effects Project

Thanks

[emcodem]

First you need to calculate the correct foldername, for that $regext is your friend.

In a populate variable processor, create a new user variable named s_project_id or such and set it to:

Code: Select all

$regext("%s_original_name%","(......)_")

This would retrieve the first 6 chars e.g. "CP1234"

Then another new variable e.g. "s_project_title" to

Code: Select all

$regext("%s_original_name%","......_(.*?)$")

This would retrieve everything after the first 6 chars and one underscore, e.g. "After_Effects_Project_File"

You can then use your variables for checking folder existence. Now it depends where is the "root" folder of the directories you want to find.

Example, hardcoded root folder:

Code: Select all

$exists("\\server\share\Root Folder\%s_project_id%_Campaign_Edit_Name")

Note that you didnt give us a hint where you take the part "Campaign_Edit_Name" from - it was not part of your filename example.

Example, root folder in the same folder as the aep file: just replace the term "\\server\share\Root Folder\" by %s_original_directory% in above example.

About checking existence of directories: Do you actually have to check if the folder exists? If not, the deliver processor would just create it in case it does not already exist.

If you need to check if the folder exists: We recently had the "folder exists" topic here:
viewtopic.php?f=5&t=1443#p8141

If the folders are never empty, i'd recommend to just use $exists function in conditional processor to check if filecount > 0.

[graham728]

Thanks for that. I think I understand most if not all of it.

The project folder will be created by a producer. So they grab the next CP tracking number then use this to name the root folder followed by the project/campaign name. There isnt a pre-definded list of project names ffastrans could look at. How the project name is entered in post the tracking number is down to he producer. I was hoping that as long as the tracker number at the start can be found that would be enough?

From reading your post this wouldn't be possible? I would need to give ffastrans a folder name post Project ID.

[ThomasM]

Code: Select all

$regext("%s_original_name%","(......)_")

@emcodem Whoppa.... what about

Code: Select all

^.{6}_

as RegEx? wouldn´t it do the job? Just wondering....


@Graham So, if I understand it correct - the Artists are putting AE / Anything from NLE and such into the Watchfolder with this unique PreFix - Right? and the Producer then creates a Folder when he´s ready for it - Right? In this case a Conditional-Node could check for existing folder with UserVariable set to emcodem´s settings. IfExist=0 then dispel Job. Set Watchfolder to "Forget Missing Files" and it will try as long as no Folder is there. If the Folder is Created by the Producer, FFAStrans is beginning it´s magic...

Have not testing this so far... So, try with caution....

cheers,
thomas

[graham728]

Hi,

Yeah so the CP1234 folder is the first thing that is created before any After Effects/Premiere Projects are created. In almost all cases the Project folder will be there. Editors add this CP1234 tracking number to the start of project files and other exports. This makes it easy to search and find assets in the future.

However, the project folder title after the CP1234 may not match what the editor has called their project. I've attached a screen grab as an example. I'm hoping there could be a way that as long as there's a folder starting with the same CP number the file can be placed within here.

This watch folder could be used for any media assets a remote editor may upload. I'm just using an After Effects project as an example. It could be an MP4 that starts with CP1234 or a .mov ProRes file.

[ThomasM]

Ah - I see. So you are good to go with the solution by @emcodem. That´s always smart

I always check RegEx´s with a RegEx-tester, e.g. at https://regex101.com/. That´s a good way to understand the functions.


So - with @emcodems solution you will have both parts of the foldername as Variables

- the six preceding [CP????] plus [_] in RegEx

Code: Select all

^.{6}_

- the Name of the Project in RegEx

Code: Select all

.{6}_(.*?)$

So, as long as the CP[NUMBER] is uniqe, the task should be solvable - just head over to the Thread @emcodem recommended earlier. A good starting point is also the Documentation on functions in FFAStrans.

Hope this helps...

Thomas

[emcodem]

OK so we use CP1234_After_Effects_Project_File.aep as example.

As far as i understood, whatever comes after CP number is not of interest, we just need the CP number and the extension for our decision.
Then we want to actually "search" for a folder name that starts with CP number and if one is found, we decide by extension which subfolder of the "found folder" we take for delivery.

Now, we already know how to get the CP number, now we just need to "find" a folder starting with CP number. For this we have the Files Find plugin processor.

I created some workflow example for you, let me know if this comes close.
The only thing is that i use "%s_original_path%\..\Edits" as starting path for locating the folder with CP number. This assumes that it is exactly as in your screenshot: "Edits" and "Watchfolder" is in the same directory.


Make sure you have the Files Find plugin processor installed:

https://www.ffastrans.com/wiki/doku.php ... processors

emcodem_locate_folder.json

(13.36 KiB) Downloaded 210 times

@emcodem Whoppa.... what about

Code: Select all

^.{6}_

fantastic, absolutely right: "^.{6}" is the same as "^......" i just shy away from using fancy regex expressions generally because it can get very hard to see what it does pretty quickly. The regex operators "() . * ^ $ \d" cover 99% of my usecases. As long as there is no urge to use anything else, i stick to that.
One might argue that it is not possible to see that "......" searches for 6 characters but my argument is that "......" makes clear that we search for "a sequence of characters" - dot is the most used regex functionality so everyone knows what it does. In difference to {6}, i needed to check this in the regex documentation. It even gets more complex if we actually search for the character {, e.g.
"^......\{6\}_" is faster to undo in my head than
"^.{6}\{6\}_"

[graham728]

Hi,

Thanks! - This works perfectly. The only issue - and this is just down to myself not fully understanding the functions. If I move the watch folder one level down into an extra watch folder I can't appear to modify the workflow correctly to do this.

\\Watch Folder > Sorting Watch Folder > (picks up file here).

I'm guessing it's something very simple. I can change it by just pointing the Find Files Node to the Edits folder (full path) and not the Original Path \..\Edits you have set up for me.

I'm guessing it's something to do with the \..\ section?

[emcodem]

The \..\ just means one folder up, it's not a ffastrans thing but a windows thing.
E.g. you have a file in C:\temp\test.txt and there is also a folder c:\temp\sub1

Then in notepad you could open exactly c:\temp\test.txt or you could open c:\temp\sub1\..\test.txt. Both point to the same file. We could also go 2 or more folders up by just repeating .. like c:\temp\sub1\sub2\..\..\test.txt and so on...

So if your workflow starts with
\\server\share\watchfolder\sorting watch folder\test.txt

and the target folder is

\\server\share\edits\subfolder1

then from perspective of test.txt we have to go 2 folders up and then into subfolder1, making the target path look like:
\\server\share\watchfolder\sorting watch folder\..\..\subfolder1

and as we have "\\server\share\watchfolder\sorting watch folder\" in the variable "%s_original_path%", in a ffastrans workflow the target folder looks like:
%s_original_path%\..\..\subfolder1

Of course nothing speaks against just writing the full path into the files find processor, it is just an advanced technique to use relative paths instead of absolute paths because this way the workflow gets portable and works with different source paths which makes testing and maintenance easier. E.g. for testing you can just copy the workflow and test with a folder structure that is on e:\test instead of on c:\productive or similar

[graham728]

Works perfectly now - thanks for taking the time to explain all of this and help me out. Appreciated!

This will massively help out sorting files out to various project folders.