ML
    • Recent
    • Categories
    • Tags
    • Popular
    • Users
    • Groups
    • Register
    • Login

    Are There Reasonable Multi-Master Over the WAN Storage Options?

    IT Discussion
    storage
    10
    45
    6.6k
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DashrenderD
      Dashrender @scottalanmiller
      last edited by

      @scottalanmiller said:

      @dafyre said:

      Why not go with Self-Hosting and replicated off-site?

      If somebody in France needs access to a file from Malaysia, then they should connect to the Malaysia file server via <insert your method here> to access the files.

      Like @scottalanmiller -- sometimes you have to change the processes.

      That's what I am thinking. I want to look at using Exablox, one at each site. Each site with their own share of which they are the master that then replicates to the other sites.

      Unless I missed it, we still haven't been told how large the files are. Is it better to deal with possible sync issues, or how about using RDS instead? If you really need to work on a file at relative performance, an RDS server in each location that users can share might be a better option.

      1 Reply Last reply Reply Quote 1
      • scottalanmillerS
        scottalanmiller
        last edited by

        RDS to a central location with VDI is being proposed as a long term solution, but not something that they are prepared to deal with in the short term.

        DashrenderD 1 Reply Last reply Reply Quote 1
        • DashrenderD
          Dashrender
          last edited by

          How does using Exablox solve a file versioning problem? What is the solution for that specific problem, assuming you can't force a lock out to all nodes?

          scottalanmillerS 1 Reply Last reply Reply Quote 0
          • scottalanmillerS
            scottalanmiller @Dashrender
            last edited by

            @Dashrender said:

            How does using Exablox solve a file versioning problem?

            Single site masters.

            DashrenderD 1 Reply Last reply Reply Quote 0
            • DashrenderD
              Dashrender @scottalanmiller
              last edited by

              @scottalanmiller said:

              RDS to a central location with VDI is being proposed as a long term solution, but not something that they are prepared to deal with in the short term.

              Just before reading this, that is exactly where my mind leapt. Centralize the whole thing - RDS to a box near that storage pool. Problem solved.

              1 Reply Last reply Reply Quote 1
              • DashrenderD
                Dashrender @scottalanmiller
                last edited by

                @scottalanmiller said:

                @Dashrender said:

                How does using Exablox solve a file versioning problem?

                Single site masters.

                Please provide more details.

                scottalanmillerS 1 Reply Last reply Reply Quote 0
                • scottalanmillerS
                  scottalanmiller @Dashrender
                  last edited by

                  @Dashrender said:

                  @scottalanmiller said:

                  @Dashrender said:

                  How does using Exablox solve a file versioning problem?

                  Single site masters.

                  Please provide more details.

                  UK Exablox with a share of which it is the master. Replication of that data goes to France and KL.

                  France Exablox with a share of which it is the master. Replication of that data goes to UK and KL.

                  KL Exablox with a share of which it is the master. Replication of that data goes to UK and France.

                  Each site gets its own local data of which it is "in charge". It has the "write" share for that data. The replication is purely for reads.

                  Each site can work with its local data as normal. It's just a normal mapped drive for them. If a site needs data from another site it grabs a read only copy super fast from the replication, makes changes and then saves those changes over the WAN to the location where the master for that share is. Cumbersome on the less common saves, but only one master for every file.

                  JaredBuschJ 1 Reply Last reply Reply Quote 3
                  • DashrenderD
                    Dashrender
                    last edited by

                    Awesome - thanks!

                    1 Reply Last reply Reply Quote 0
                    • JaredBuschJ
                      JaredBusch @scottalanmiller
                      last edited by

                      @scottalanmiller said:

                      @Dashrender said:

                      @scottalanmiller said:

                      @Dashrender said:

                      How does using Exablox solve a file versioning problem?

                      Single site masters.

                      Please provide more details.

                      UK Exablox with a share of which it is the master. Replication of that data goes to France and KL.

                      France Exablox with a share of which it is the master. Replication of that data goes to UK and KL.

                      KL Exablox with a share of which it is the master. Replication of that data goes to UK and France.

                      Each site gets its own local data of which it is "in charge". It has the "write" share for that data. The replication is purely for reads.

                      Each site can work with its local data as normal. It's just a normal mapped drive for them. If a site needs data from another site it grabs a read only copy super fast from the replication, makes changes and then saves those changes over the WAN to the location where the master for that share is. Cumbersome on the less common saves, but only one master for every file.

                      Exactly, and if they restructure their shares correctly, those less common saves should really be uncommon.

                      scottalanmillerS StefUkS 2 Replies Last reply Reply Quote 2
                      • StefUkS
                        StefUk
                        last edited by

                        Hi everyone,
                        thanks for chiming in ... and thanks @scottalanmiller for posting this on my behalf.

                        RDS is currently out of the question at the moment due to intense graphical resources that they need. We are looking at some long term solutions - Nvidia Grid of some sort but at the moment RDS will not cut it.

                        stef

                        1 Reply Last reply Reply Quote 1
                        • scottalanmillerS
                          scottalanmiller @JaredBusch
                          last edited by

                          @JaredBusch That's what I am thinking. Hopefully the workflow will make for a pretty solid local storage situation the bulk of the time.

                          1 Reply Last reply Reply Quote 1
                          • StefUkS
                            StefUk @JaredBusch
                            last edited by

                            @JaredBusch @JaredBusch @scottalanmiller

                            that's good in terms of replication.
                            but what about working on the same files - project ? what you are saying is that there is no way to get them to work on the same project without file version issues ?

                            Stef

                            scottalanmillerS 1 Reply Last reply Reply Quote 0
                            • scottalanmillerS
                              scottalanmiller @StefUk
                              last edited by

                              @StefUk said:

                              @JaredBusch @JaredBusch @scottalanmiller

                              that's good in terms of replication.
                              but what about working on the same files - project ? what you are saying is that there is no way to get them to work on the same project without file version issues ?

                              Correct, there is not. If they each have a copy of their own data, they each have an opportunity to work on them at the same time. Once you have multiple masters, you have issues. No way around that.

                              wirestyle22W 1 Reply Last reply Reply Quote 2
                              • wirestyle22W
                                wirestyle22 @scottalanmiller
                                last edited by

                                @scottalanmiller said:

                                @StefUk said:

                                @JaredBusch @JaredBusch @scottalanmiller

                                that's good in terms of replication.
                                but what about working on the same files - project ? what you are saying is that there is no way to get them to work on the same project without file version issues ?

                                Correct, there is not. If they each have a copy of their own data, they each have an opportunity to work on them at the same time. Once you have multiple masters, you have issues. No way around that.

                                So really we're trying to figure out how to combine all of the changes, correct? Can't this be done with .tmp files fragmenting and then recombining? I'm sure their software doesn't support this but I'm just asking hypothetically for my own knowledge.

                                scottalanmillerS 1 Reply Last reply Reply Quote 0
                                • scottalanmillerS
                                  scottalanmiller @wirestyle22
                                  last edited by

                                  @wirestyle22 said:

                                  So really we're trying to figure out how to combine all of the changes, correct? Can't this be done with .tmp files fragmenting and then recombining?

                                  Not generically, no. Combining changes is never something that can be handled by storage. An application might be able to do that, but a storage system never can.

                                  wirestyle22W 1 Reply Last reply Reply Quote 0
                                  • wirestyle22W
                                    wirestyle22 @scottalanmiller
                                    last edited by wirestyle22

                                    @scottalanmiller said:

                                    @wirestyle22 said:

                                    So really we're trying to figure out how to combine all of the changes, correct? Can't this be done with .tmp files fragmenting and then recombining?

                                    Not generically, no. Combining changes is never something that can be handled by storage. An application might be able to do that, but a storage system never can.

                                    Yeah I mean at the application level. She would need a third party piece of software that specifically handles this--which is another point of failure 😞

                                    1 Reply Last reply Reply Quote 1
                                    • Deleted74295D
                                      Deleted74295 Banned
                                      last edited by

                                      Hmm, I can't remember exactly but I think the BBC actually pulled this off with a particular product.

                                      I can't remember what it's called.

                                      scottalanmillerS 1 Reply Last reply Reply Quote 0
                                      • scottalanmillerS
                                        scottalanmiller @Deleted74295
                                        last edited by

                                        @Breffni-Potter said:

                                        Hmm, I can't remember exactly but I think the BBC actually pulled this off with a particular product.

                                        I can't remember what it's called.

                                        You can do it, like I can build it with GFS2 and DRBD, the problem is once a WAN link fails you have a disaster. Do you simply cut everyone off? Or do you allow local edits?

                                        Deleted74295D 1 Reply Last reply Reply Quote 1
                                        • scottalanmillerS
                                          scottalanmiller
                                          last edited by

                                          If you have the budget to go for "never fail" WAN links, you can do a lot to make this kind of thing feasible. But when they do fail, you have to come up with a failure mitigation strategy.

                                          1 Reply Last reply Reply Quote 1
                                          • Deleted74295D
                                            Deleted74295 Banned @scottalanmiller
                                            last edited by Deleted74295

                                            @scottalanmiller said:

                                            @Breffni-Potter said:

                                            Hmm, I can't remember exactly but I think the BBC actually pulled this off with a particular product.

                                            I can't remember what it's called.

                                            You can do it, like I can build it with GFS2 and DRBD, the problem is once a WAN link fails you have a disaster. Do you simply cut everyone off? Or do you allow local edits?

                                            No I mean actually sorting it despite that.

                                            Done some googling to see if I can find the product but can't but effectively, the storage at each location is irrelevant, what you do is use a digital asset management system.

                                            This does your versioning of media files, checking, quality control. It's a very different way of working to "everyone throw your data into a file browser"

                                            wirestyle22W scottalanmillerS 2 Replies Last reply Reply Quote 0
                                            • 1
                                            • 2
                                            • 3
                                            • 2 / 3
                                            • First post
                                              Last post