Are There Reasonable Multi-Master Over the WAN Storage Options?
-
Hi everyone,
thanks for chiming in ... and thanks @scottalanmiller for posting this on my behalf.RDS is currently out of the question at the moment due to intense graphical resources that they need. We are looking at some long term solutions - Nvidia Grid of some sort but at the moment RDS will not cut it.
stef
-
@JaredBusch That's what I am thinking. Hopefully the workflow will make for a pretty solid local storage situation the bulk of the time.
-
@JaredBusch @JaredBusch @scottalanmiller
that's good in terms of replication.
but what about working on the same files - project ? what you are saying is that there is no way to get them to work on the same project without file version issues ?Stef
-
@StefUk said:
@JaredBusch @JaredBusch @scottalanmiller
that's good in terms of replication.
but what about working on the same files - project ? what you are saying is that there is no way to get them to work on the same project without file version issues ?Correct, there is not. If they each have a copy of their own data, they each have an opportunity to work on them at the same time. Once you have multiple masters, you have issues. No way around that.
-
@scottalanmiller said:
@StefUk said:
@JaredBusch @JaredBusch @scottalanmiller
that's good in terms of replication.
but what about working on the same files - project ? what you are saying is that there is no way to get them to work on the same project without file version issues ?Correct, there is not. If they each have a copy of their own data, they each have an opportunity to work on them at the same time. Once you have multiple masters, you have issues. No way around that.
So really we're trying to figure out how to combine all of the changes, correct? Can't this be done with .tmp files fragmenting and then recombining? I'm sure their software doesn't support this but I'm just asking hypothetically for my own knowledge.
-
@wirestyle22 said:
So really we're trying to figure out how to combine all of the changes, correct? Can't this be done with .tmp files fragmenting and then recombining?
Not generically, no. Combining changes is never something that can be handled by storage. An application might be able to do that, but a storage system never can.
-
@scottalanmiller said:
@wirestyle22 said:
So really we're trying to figure out how to combine all of the changes, correct? Can't this be done with .tmp files fragmenting and then recombining?
Not generically, no. Combining changes is never something that can be handled by storage. An application might be able to do that, but a storage system never can.
Yeah I mean at the application level. She would need a third party piece of software that specifically handles this--which is another point of failure
-
Hmm, I can't remember exactly but I think the BBC actually pulled this off with a particular product.
I can't remember what it's called.
-
@Breffni-Potter said:
Hmm, I can't remember exactly but I think the BBC actually pulled this off with a particular product.
I can't remember what it's called.
You can do it, like I can build it with GFS2 and DRBD, the problem is once a WAN link fails you have a disaster. Do you simply cut everyone off? Or do you allow local edits?
-
If you have the budget to go for "never fail" WAN links, you can do a lot to make this kind of thing feasible. But when they do fail, you have to come up with a failure mitigation strategy.
-
@scottalanmiller said:
@Breffni-Potter said:
Hmm, I can't remember exactly but I think the BBC actually pulled this off with a particular product.
I can't remember what it's called.
You can do it, like I can build it with GFS2 and DRBD, the problem is once a WAN link fails you have a disaster. Do you simply cut everyone off? Or do you allow local edits?
No I mean actually sorting it despite that.
Done some googling to see if I can find the product but can't but effectively, the storage at each location is irrelevant, what you do is use a digital asset management system.
This does your versioning of media files, checking, quality control. It's a very different way of working to "everyone throw your data into a file browser"
-
@Breffni-Potter said:
@scottalanmiller said:
@Breffni-Potter said:
Hmm, I can't remember exactly but I think the BBC actually pulled this off with a particular product.
I can't remember what it's called.
You can do it, like I can build it with GFS2 and DRBD, the problem is once a WAN link fails you have a disaster. Do you simply cut everyone off? Or do you allow local edits?
No I mean actually sorting it despite that.
Done some googling to see if I can find the product but can't but effectively, the storage at each location is ireelevant, what you do is use a digital assest management system.
This does your versioning of media files, checking, quality control. It's a very different way of working to "everyone throw your data into a file browser"
I would think it depends on the system and how it actually checks if a file is being accessed. If the WAN link is down it would most likely assume that no one else has it opened or just error out if its configured to do that if unable to connect. I haven't had to do this though.
-
@Breffni-Potter said:
This does your versioning of media files, checking, quality control. It's a very different way of working to "everyone throw your data into a file browser"
We do that with Sharepoint, but it doesn't support multi-master and it doesn't sort out conflicts.
-
@wirestyle22 said:
I would think it depends on the system and how it actually checks if a file is being accessed. If the WAN link is down it would most likely assume that no one else has it opened or just error out if its configured to do that if unable to connect. I haven't had to do this though.
That's the problem it either has to lock everyone out or allow conflicts.
-
@scottalanmiller said:
@Breffni-Potter said:
This does your versioning of media files, checking, quality control. It's a very different way of working to "everyone throw your data into a file browser"
We do that with Sharepoint, but it doesn't support multi-master and it doesn't sort out conflicts.
But certain DAM packages do that. it's built specifically for digital assets which are constantly being edited/changed.
-
They are stored as the same file but can see the different versions, which user worked on which version, ect.
-
@Breffni-Potter said:
They are stored as the same file but can see the different versions, which user worked on which version, ect.
My multi-master system does that naturally. In both cases, humans have to sort it out later.
-
@Breffni-Potter said:
They are stored as the same file but can see the different versions, which user worked on which version, ect.
Man, I can see all kinds of problems with that system. and as Scott said, it has to be sorted by a person.
-
@Dashrender said:
@Breffni-Potter said:
They are stored as the same file but can see the different versions, which user worked on which version, ect.
Man, I can see all kinds of problems with that system. and as Scott said, it has to be sorted by a person.
Yeah. All the workable solutions I've seen involve change tracking of some sort.
-
There are systems that do this with version control. You could do this with GIT or Mercurial, in theory. But eventually a human always has to sort things out.
