Dailies creation error

Customize rules.
User avatar
priel71
Posts: 63
Joined: Fri Jan 20, 2017 2:10 am
Location: Kuala Lumpur
Contact:

Dailies creation error

Post by priel71 »

Hi,
I just upgraded our CGRU install from 2.3x to the latest github master. First I tried to pull it directly from github, since my install was a clone repo. While that worked fine, I did have an issue with the dailies creation. On any project I had, when I sent a daily request to Afanasy, the render failed with the below error:

MakeMovie: "/mnt/shared/cgru/utilities/moviemaker/makemovie.py"
/RAJAH/SCENES/sc002/RAJA_002_040/RESULT/DPX/v007
Can't find input directory "/RAJAH/SCENES/sc002/RAJA_002_040/RESULT/DPX"


So I thought maybe some of my local commits had screwed up the merge.. therefore I did a clean repo clone and moved over my settings in all the config files by hand. Still the same problem.

I'm not sure where to look, since my setup has been working flawlessly for the past years with these config settings. Converting to sequences/movies work fine from RULES, only the dailies.

Any ideas of where to start my troubleshooting would be appreciated.

Peter.
User avatar
priel71
Posts: 63
Joined: Fri Jan 20, 2017 2:10 am
Location: Kuala Lumpur
Contact:

Re: Dailies creation error

Post by priel71 »

After some digging around... this is the string sent from RULES to AFANASY:

Code: Select all

#36 16:26:19.917: send 11 (1) : {"afanasy":1,"job":{"name":"RAJA_002_040_v007_200517","folders":{"input":"/RAJAH/SCENES/sc002/RAJA_002_040/RESULT/DPX","output":"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_040/RESULT/DAILIES","rules":"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_040"},"blocks":[{"name":"Dailies","service":"movgen","parser":"generic","capacity":500,"working_directory":"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_040","tasks":[{"name":"RAJA_002_040_v007_200517","command":"python \"/mnt/shared/cgru/utilities/moviemaker/makemovie.py\" -a \"ffmpeg\" -c \"h264_high\" -f 24 -r 2880x2160 -s dailies_slate -t dailies_withlogo --font \"DejaVuSans\" --colorspace \"Log\" --audio \"REF/sound.flac\" --lgspath \"bcvfx_frame.png\" --lgssize 30 --lgsgrav SouthEast --lgfpath \"logo.png\" --lgfsize 10 --lgfgrav North --project \"RAJAH\" --shot \"RAJA_002_040\" --ver \"v007\" --artist \"Bon\" --activity \"Comp\" --pcodec \"h264_mid\" --pargs \"-vf scale=640:-1\" --pdir \".rules\" --aspect_in 2 --createoutdir \"/RAJAH/SCENES/sc002/RAJA_002_040/RESULT/DPX/v007\" \"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_040/RESULT/DAILIES/RAJA_002_040_v007_200517\""}]}],"user_name":"peter","host_name":"chrome","branch":"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_040/RESULT/DAILIES"},"address":"192.168.100.100","port":51000,"sender_id":0,"magick_number":1}
I'm not sure if the "input" are supposed to have the full path, because as far as I can see that's the only issue.. the rest seem to solve fine.

If it's not clear from the above, I have a system wide cgru folder at /mnt/shared/cgru and a project folder on /mnt/projects which is linked into the cgru folder.

Peter.
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Dailies creation error

Post by timurhai »

Hi!

All paths should be correct (full), all folders should exist in the file system.

May be you have onvalid Rules or (and) paths map setup.
Have you override this rule to your "/mnt/projects":
https://github.com/CGRU/cgru/blob/maste ... al.json#L3

In Rules, when you navigating some folder, path in the bottom info line is correct?

Can you paste here dailies job task command here?
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
User avatar
priel71
Posts: 63
Joined: Fri Jan 20, 2017 2:10 am
Location: Kuala Lumpur
Contact:

Re: Dailies creation error

Post by priel71 »

Hi Timur, thank you for your reply!

Yes, I can see at the task bar that it does resolve correctly to /mnt/projects/xxx/xx/xx/xx
I can also open vlc, djv_view etc directly from RULES without any problem. Walk works fine as well.

I'm not sure but I guess you want the command I can see in the log when I send off a daily yes?

If so, here is the output:

Code: Select all

#3 20:06:42.513: send 18 (1) : {"afanasy":1,"job":{"name":"RAJA_002_020_v010_200518","folders":{"input":"/RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DPX","output":"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DAILIES","rules":"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_020"},"blocks":[{"name":"Dailies","service":"movgen","parser":"generic","capacity":500,"working_directory":"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_020","tasks":[{"name":"RAJA_002_020_v010_200518","command":"python \"/mnt/shared/cgru/utilities/moviemaker/makemovie.py\" -a \"ffmpeg\" -c \"h264_high\" -f 24 -r 2880x2160 -s dailies_slate -t dailies_withlogo --font \"DejaVuSans\" --colorspace \"Log\" --audio \"REF/sound.flac\" --lgspath \"bcvfx_frame.png\" --lgssize 30 --lgsgrav SouthEast --lgfpath \"logo.png\" --lgfsize 10 --lgfgrav North --project \"RAJAH\" --shot \"RAJA_002_020\" --ver \"v010\" --artist \"Bon\" --activity \"Comp\" --pcodec \"h264_mid\" --pargs \"-vf scale=640:-1\" --pdir \".rules\" --aspect_in 2 --createoutdir \"/RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DPX/v010\" \"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DAILIES/RAJA_002_020_v010_200518\""}]}],"user_name":"peter","host_name":"chrome","branch":"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DAILIES"},"address":"192.168.100.100","port":51000,"sender_id":0,"magick_number":1}
#2 20:06:36.312: recv 17 (0) walk dailies : received
#1 20:06:36.206: send 17 (1) walk dailies: {"cmdexec":{"cmds":["rules/bin/walk.sh -m \"projects/RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DPX/v010\""]}}
#0 20:06:36.205: Make Dailies: /RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DPX/v010
I have also attached a screenshot of how the daily GUI looks before I send.

Thanks,

Peter.
Attachments
Annotation 2020-05-18 201009.png
Annotation 2020-05-18 201009.png (94.2 KiB) Viewed 9717 times
User avatar
priel71
Posts: 63
Joined: Fri Jan 20, 2017 2:10 am
Location: Kuala Lumpur
Contact:

Re: Dailies creation error

Post by priel71 »

Or maybe you meant this, from the job on AFANASY:

Code: Select all

{
  "name": "RAJA_002_020_v010_200518",
  "service": "movgen",
  "capacity": 500,
  "time_start": 1589808692,
  "job_id": 7,
  "block_num": 0,
  "task_num": 0,
  "command_block": "",
  "command_task": "python \"/mnt/shared/cgru/utilities/moviemaker/makemovie.py\" -a \"ffmpeg\" -c \"h264_high\" -f 24 -r 2880x2160 -s dailies_slate -t dailies_withlogo --font \"DejaVuSans\" --colorspace \"Log\" --audio \"REF/sound.flac\" --lgspath \"bcvfx_frame.png\" --lgssize 30 --lgsgrav SouthEast --lgfpath \"logo.png\" --lgfsize 10 --lgfgrav North --project \"RAJAH\" --shot \"RAJA_002_020\" --ver \"v010\" --artist \"Bon\" --activity \"Comp\" --pcodec \"h264_mid\" --pargs \"-vf scale=640:-1\" --pdir \".rules\" --aspect_in 2 --createoutdir \"/RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DPX/v010\" \"/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DAILIES/RAJA_002_020_v010_200518\"",
  "frame_start": 0,
  "frame_finish": 0,
  "frames_inc": 1,
  "frames_num": 1,
  "parser": "generic",
  "working_directory": "/mnt/projects/RAJAH/SCENES/sc002/RAJA_002_020"
}
Looking at the above, it seems it's not detecting the frame sequence..?
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Dailies creation error

Post by timurhai »

Yes. Is there a frame sequence? How it looks like (name, dot, frame with padding, dot, extension)?
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
User avatar
priel71
Posts: 63
Joined: Fri Jan 20, 2017 2:10 am
Location: Kuala Lumpur
Contact:

Re: Dailies creation error

Post by priel71 »

Looks like this:

RAJA_002_020_01418099.dpx - RAJA_002_020_01418276.dpx

But these are the same sequences that worked fine in RULES before I did the upgrade..

Peter.
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Dailies creation error

Post by timurhai »

#2 20:06:36.312: recv 17 (0) walk dailies : received
#1 20:06:36.206: send 17 (1) walk dailies: {"cmdexec":{"cmds":["rules/bin/walk.sh -m \"projects/RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DPX/v010\""]}}
#0 20:06:36.205: Make Dailies: /RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DPX/v010

- open log (left bottom button with an arrow)
- turn on Responces (click the button)
- then you will see what was received on "walk dailies" request
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
User avatar
priel71
Posts: 63
Joined: Fri Jan 20, 2017 2:10 am
Location: Kuala Lumpur
Contact:

Re: Dailies creation error

Post by priel71 »

Interesting, it does seem to find 179 files, which is correct, that is the amount of dpx files in that folder.. Well, actually 178, but I guess the .rules folders counts as 1 as well.

Code: Select all

#2 21:55:13.498: recv 70 (0) walk dailies : {"cmdexec":[{"walk":{"path":"projects\/RAJAH\/SCENES\/sc002\/RAJA_002_020\/RESULT\/DPX\/v010","time_start":"Mon May 18 21:55:13 2020","previous":" 179 files, 1 folders, 6 645 272 989 bytes","time_finish":"Mon May 18 21:55:13 2020","processed":" 179 files, 1 folders, 6 645 272 989 bytes","walk":{"folders":{".rules":{"num_files_total":1,"size_total":413,"space":4096,"num_images":0,"num_folders":0,"num_files":1,"num_folders_total":0,"size":413}},"files":[],"size_total":6645272989,"space":6645641216,"num_images":178,"num_folders":1,"num_files":178,"num_files_total":179,"num_folders_total":1,"size":6645272576},"delta":"0 files, 0 folders, 0 bytes","time_run":"00:00:00.031","status":"success"}}]}
#1 21:55:13.287: send 70 (1) walk dailies: {"cmdexec":{"cmds":["rules/bin/walk.sh -m \"projects/RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DPX/v010\""]}}
#0 21:55:13.285: Make Dailies: /RAJAH/SCENES/sc002/RAJA_002_020/RESULT/DPX/v010
User avatar
timurhai
Site Admin
Posts: 911
Joined: Sun Jan 15, 2017 8:40 pm
Location: Russia, Korolev
Contact:

Re: Dailies creation error

Post by timurhai »

Hm. Rules (walk command) can't find any files in that folder: "files":[]
Timur Hairulin
CGRU 3.3.1, Ubuntu 20.04, 22.04, MS Windows 10 (clients only).
Post Reply