“That’s when we first noticed it, with Woody.”
“[Larry Cutler] was in that directory and happened to be talking about installing a fix to Woody or Woody’s hat. He looked at the directory and it had like 40 files, and he looked again and it had four files.”
“Then we saw sequences start to vanish as well and we were like, “Oh my god”
“I grabbed the phone…unplug the machine!””
That’s Oren Jacob, former Chief Technical Officer of Pixar—then an associate technical director for Toy Story 2—recounting the moment they discovered that the movie was being deleted off of the company’s servers after an erroneous command was executed, erasing two months and hundreds of man-hours worth of work.
You might have heard something about this lately, as a clip from the special features of the movie has been making the rounds after being posted on Tested. It’s narrated by Jacob himself, and the movie’s Supervising Technical Director Galyn Susman.
“You have 400 people on the network and they all have to have like pretty massive access across the board to the whole project, so it’s hard to like, limit the damage,” Jacob said. “It could happen from almost any terminal.”
“Pixar being a wide open Unix environment meant that it was very promiscuous. You could [change directory] ‘slash’, net ‘slash’ or walk across the network and log into Ed Catmull’s machine or Steve Job’s machine if you wanted to. Not that Steve ever did do any work on the film directly, but you could do that.”
The common way to prevent an accidental command like this being run on an entire project is to lock users down with permissions to only the files they need. But, because of the way a project like a Pixar film works, almost everyone working on the show needed permissions to read and write to the master machine. This was their job.
Assigning micro-managed permissions would have eaten up administrative resources, especially in crunch time.
This sort of problem is exactly what EditShare shared storage was designed to solve in the first place.
With EditShare, film & video editors can collaborate on footage on a common file server and be assured that situations like this shouldn’t be a problem, because the system actively manages permissions so that simple human mistakes don’t become catastrophes.
It works great, and we’d love to talk to you about it if you’d like to learn more.