Get the Most Out of GoAnywhere: High Availability and Clustering

Chris Spargen: Thanks for everyone that's joined today. Specifically we're going to focus today on getting the most out of high availability and clustering. We'll talk about some of the pros to doing that. I wanted to touch base on a couple of key items. We did schedule this event for a half an hour. In the event you have to leave early, we are recording it, so if you need to rewatch anything or you want to share it with a friend or a colleague, you can definitely do that within a day after the event, we'll send out a link so you have that. Any questions that you have throughout the webinar, go ahead and submit those through the Q&A pane. We'll have a couple team members online that will be able to answer those throughout the presentation. Then at the end of the webinar, we usually try to squeeze in some Q&A time as well. That way if you want to stay online and submit a question live, you can do that.

Then at the end of the webinar you're going to see a quick survey that'll pop up. Please fill that out. We really value feedback and want to create the best processes in place that we can. If you have any questions that are answered on the call as well, you can also enter those at that time and then someone will get back to you. Thanks again for joining and we appreciate it.

All right. The agenda today. If you're new, we're going to touch base quickly on what is GoAnywhere MFT. I think the majority of you are existing customers and are using GoAnywhere, so you should have a pretty good grasp on what GoAnywhere does, how you're using it, et cetera. We'll roll then into why would you cluster GoAnywhere MFT? We'll touch on the high availability of clustering as well as the load balancing capabilities. Then we'll walk through a live demo of this and then at the end we'll have that Q&A session.

Today your presenter, if you've attended some of these in the past, you might recognize the voice of Dan Freeman. That is not me. My name is Chris Spargen. I'm the support manager. I've been working on the GoAnywhere product for about five years. It's been really, really fascinating to watch it grow. All the new functionality that we've added in that timeframe has been really interesting. I've done a mixed bag of support as well as professional services and liaison for a bit between support and development. Now I head up the support team. Lots of good stuff. It's a really, really fun application that I enjoy supporting. Every day is a little different, so keeps me on my toes.

What is GoAnywhere MFT?

What is GoAnywhere MFT? I would define GoAnywhere MFT as a holistic managed file transfer solution. Recently, some of our functionality we've added has even gotten outside of just file transfers but also included some layers of business automation as well. Think of MFT as an application that can handle inbound file transfers and automate the processes involved with that.

Let's say a trading partner sends you a file that you need to decrypt and you also need to parse the file and you need to upload it to your backend system, maybe that's a database. Then send an email notification to someone notifying them that that process is complete. All those things are wrapped into GoAnywhere MFT. I'd say more specifically over the course of the last five to seven years, the security's become more hot.

We have incorporated a ton of security checks in this whole process as well. Maybe that file, after you decrypt it, you need to read it against an antivirus or a DLP or data loss prevention service to make sure that it doesn't contain any content before it moves into that next stage of the workflow process. That's one example of things that we've added.

We've got great auditing on all the activity that takes place in the application as well. Hopefully you've been reading some of the points on the screen here. We've also incorporated some cloud functionality as well in the last year and a half or so. A ton of different things you can do with this product. I would say no customer is using it in the same fashion as another. There's just so many different ways that you can integrate this product. As I said earlier, super fun to support and watch grow.

Why Cluster GoAnywhere MFT

Moving into, why would you cluster GoAnywhere MFT? Some of the bullet points you'll read here on the screen. The main two topics would be focused on high availability and load balancing. Anytime that you've got any mission critical file transfers or workflows that are in GoAnywhere MFT, let's say a VM that's running, you're going to run instant, takes a dive. I'm sure that's never happened to anyone on this call. In the event something like that happens, you can have another instance of GoAnywhere that is working in tandem. We call those nodes in a cluster. That node can take over and continue to execute those mission critical transfers and ensure that you don't have downtime. If you've got SLAs in place with any customers or trading partners, that might be another reason that you'd consider high availability an important piece of this. We find that it typically increases our user satisfaction from the sake of... I guess it provides you peace of mind just knowing that in the event something does go awry, GoAnywhere is still going to be operating and those transfers are still going to take place.

Then finally, some of you may have compliance requirements that force you to prove that you've got a application that will maintain and continue in the event of disaster. The other reason that we find a lot of customers want to cluster GoAnywhere is for the load balancing piece. If you've got two nodes in your GoAnywhere cluster and you've got connections that are coming into, say your SFTP service, GoAnywhere has the built in load balancing capability to take each connection and round robin that to each node. This is going to give you much, much I guess more bandwidth and the workload that GoAnywhere can take on is now doubled. Lots of inbound transfers. This is also a good idea.

Then we also have your workflows that are scheduled. You have the capability to distribute those, so each node of your cluster can shoulder again a higher workload. Then in the event let's say, there I use the VM crashing as an example. Let's say that happens. The load balancing is smart enough to know that, "Okay, now the traffic goes to my one remaining node inside the cluster." Those are the two main reasons.

High Availability with GoAnywhere

Thought I'd show a couple just diagrams here that may help see what this looks like. If we've got two GoAnywhere MFT instances running in a cluster, then you've got a shared database and shared file system that the two nodes will use. This is going to be their way of communicating and not duplicating any work. You can see there in the snippets we support all the major players for the shared database. As far as file systems go, typically you'd see something like a Windows file share with DFS running that's replicating that in the event that, let's say your file system goes down and you've got that available as well.

Load Balancing with GoAnywhere

We also see, on Linux systems typical to see an NFS mount or SIFS Mount in place. Again, that file system shared for the sake of those nodes being able to see their central area where all their files, documents, et cetera are stored. Another diagram from a load balancing perspective is going to show you two GoAnywhere MFT nodes on the right clustered as we just discussed. They're going to share the file system in the database. Those are going to be in your internal network. Your DMZ would have GoAnywhere Gateway.

Hopefully everyone on this call was either using or looking into GoAnywhere Gateway solely for the sake of stopping all that inbound traffic in the DMZ. Not allowing any traffic into the internal network, but GoAnywhere Gateway is your reverse proxy and your load balancer. Let's say that top node crashed. The VM went down, some hardware failure occurred, maybe a natural disaster like the hurricane that is on the East coast right now happens and production system two is our only one left. GoAnywhere Gateway knows to route that traffic to the production system to maintain the high availability. But in a normal scenario, where they're both operating and functioning, we're going to load balance those sessions that come in through the Gateway to both of those nodes of GoAnywhere MFT. That's a diagram overview of what clustering looks like.

Live Demonstration of GoAnywhere

Let's move into... I'm going to do a live demo here of setting up clustering in GoAnywhere, just to show you how easy it is. I think that's probably one of the main takeaways in supporting this product that I would say a lot of customers enjoy. It's pretty simple to set up clustering. I've got two GoAnywhere instances here. Let me go ahead and start this one here. I've got one standalone instance of MFT that is brand new, freshly installed. If we go over to web users, you can see there's nothing here. If we look in our projects, nothing at all. This is a default standard instance of GoAnywhere MFT. If we look at this other instance here that I'm bringing up, this is a GoAnywhere instance that you would compare to your own. Where you've got all your mission critical file transfers, your web users set up, all that good stuff.

If we look over here at this instance here, it's populated. I've already followed some best practice recommendations, in that I'm pointing to a shared file system for like a DR server to be able to use. If we look at the database configuration I'm already using, a backend database that is not the embedded Derby database. For those of you that are on this call that are still using that, I would typically say we include that for the purpose of easy trial. We do support all the major players. I guess if you can pair DB2, Oracle, Microsoft SQL or MySQL to Apache Derby, like comparing maybe a Kia to a BMW. They're both going to get you from A to B, but if you had a choice and money was no object, you probably would opt for the later.

That's one step to the clustering process. As we said, there's a shared database here. In this case, I'm already running on that. All that's really necessary for the clustering piece is... I had our install guide here because I think that as you go to set this up for the first time, this is a really great resource for you. Not only do we talk about clustering, this would be on page 49 of our install guide, the most recent one, but we walked through all the steps, step-by-step. The main thing that you need to first start up doing is make sure you're externalized. Make sure you're licensed for clustering, which you can access from our customer portal. As we talked about, because you have to have that shared file system in place, you're going to need to point GoAnywhere MFT to an external file server.

Occasionally we'll see file systems that are local and then maybe replicated or maybe one of the nodes has a file system that's shared on it. I discourage that. The main reason is that if that node goes down, your other node can't operate, because that file system is no longer available for it. This really needs to be a centralized shared file server that both nodes can access.

The first step is pushing all those locations out to that file share. So in my case here, I've got... Let me go up to my log settings just to show you here. I actually am running this already on a Mount. Like I said, hopefully for most of you that are using GoAnywhere you are pointing to somewhere or replicating the file system. In this case a best practice that we typically see is if a DR instance needs to access a file system, a lot of customers will push this out to a shared location or a network file server.

That way their DR instance can access it in the event it goes down. This is an NFS Mount on a Linux machine. You do need to change all data locations. That's what I had the install guide up for. That's going to be under your domains. For each domain that you have in place, you're going to have projects, workspace, web box directories defined there. Those all need to be changed and pointed out to your shared file system. Your logs need to point out there as well, under the log settings which we can see, this is currently doing. Your system global settings data tab here, that's going to be the majority of your other directories that all need to be pointed out to that shared file system. Then finally, if you're using our legacy file-based key management and you haven't upgraded, GoAnywhere to... As a five, six we included key management, key management system here, as you can see. If you are using these older key management systems, you'd also need to point your certificate manager preferences and then your PGP keys in the preferences. All those need to be pointed out to this shared file location.

Once you have done that, you are ready to switch your database. In this case, like I said, I'm already running on my SQL. So my primary, my production instance of GoAnywhere, it actually doesn't have to... We don't have to do anything with this, because I've already externalized to a shared database. I'm ready to have any other additional nodes that I want to add to this cluster tied into that database schema, as well. This instance already prepped and ready to go. So much like hopefully lots of your deployments, you're already in a place where GoAnywhere is running externalized. If it's not pointing out to that external file system, you'll want to do that. Then you'll just install a new version of GoAnywhere MFT that is of the same version.

If you're not up on the latest and greatest versions and you don't have your initial like install file you can always contact support. We'll send you the exact version that you're currently operating on, because that is one prerequisite of clustering, is you need to be on the same versions. Occasionally we're going to change database tables or we're going to add new ones for new functionality. The product version has to be the same in order to have your cluster successfully boot up and run. That is why I'm running five, seven, four here on both.

Let's go first things first. Let's go onto my Linux machine here. Let's see, I'm in. I'm in my home install directory. The first thing you need to do is go to the config folder and inside of your config folder there is a cluster XML file. If I go ahead and use something like VI to open this up and view it, you're going to be able to see that this cluster XML. In this case I did some prep work. This cluster actually is already enabled for this particular node.

Typically, this file is going to be completely blank and this value is going to be set to false for the cluster enabled. What you need to define for this is what's your system name going to be? In this instance I called it Sento S, support node. Your cluster bind address and cluster bind port. This is going to be the local IP and port that this node in your cluster is going to be utilizing for communicating with other nodes in your cluster. You do need to make sure that this is accessible for other nodes in the cluster, because this is their way of communicating with one another.

Whatever you define for your cluster bind address and cluster bind port, just make sure that that's open and available to your other node in the cluster. Otherwise, you'll have an issue starting it up. As I said, you are going to need to set this value at the bottom to true, from false and this is what enables the cluster from the get go. We want to save those changes. Go ahead and do that. If we run that, we can see that cluster XML file 10, 20 was updated so that node is ready for clustering. If I go out now and start up this GoAnywhere... Excuse me, I need to stop it. You can make changes to that file while you're running. That won't cause any issues. If I go ahead and stop that node and start it back up, usually want to wait about five seconds to make sure everything clears out of the database table there. Go ahead and start that.

There we go. We'll load that. Our other instance here, you'd tell us the different IP address, the blank environment if you will, hasn't been modified at all. This would, as we said, we're doing a live demo of you actually doing this. This particular instance is where you see the other screen here on this VM. That's going to be what we will use.

If we go back here. You can see the login page. One thing that's going to change for you is you're going to see the system name appear when you're clustered. The environment, from the global settings that is stored in the database. That would have already been defined. The system name that you defined in that cluster XML file, that's what you're going to actually see.

If we log back in here, we've got one node up and running in our cluster. You get, when that cluster enabled is set to true in the cluster XML file, you'll get a cluster manager option here under system. We can see I've got one node right now. It is the coordinator.

When you're running in a cluster environment you have a coordinator and then all additional nodes that join, are considered participants. In there, you can also see what's that cluster address and cluster report. Then this status update, which is currently auto refreshing. You can see this is a value that's updated and tells us this instance is active in the cluster.

All right, so moving over to this other instance that I've installed and got up and running here. All you have to do with your additional instance of GoAnywhere, there is no more going into your logs and defining all these different locations. You don't have to do that on this node. The main reason for that is that that is stored in the database. Using our handy switch database wizard, hopefully some of you are familiar with this, you can tie into that backend database. If I connect here to the same database server that my other node is using, here we go, I can see I just need to replicate those settings.

Once I've done that, if you have any specific settings in your JDBC URL that maybe this instance is using, you can actually copy this over and paste it in here as well. Maybe you have some properties that are unique to your DB server. That would be the time to put that in place. If we hit next here, this is the neat part of adding additional nodes to your cluster. In my opinion, what makes it so simple, is on this option here for do nothing, the database already has valid tables, indexes and data. That's the case because this coordinator node of the cluster that we've now clustered or made cluster ready, it's already got all the data that you need. It's got all those locations pointed out. All you have to do is take that third radio option and hit next, then hit finish and there's a nice little progress bar.

Usually when you're tying in, this is extremely fast. Let's say your MFT instance has a few million records in the database. That initial node that you poured over... For those of you that have switched your database to an external database like MySQL or Microsoft SQL server, you've probably been through this where you can track the progress bar. If you're busy like most IT folks and you need to maybe check email or put out a fire while this process completes, there is a nice progress bar for you. In this case since I'm just tying into it, there's not much to that and it's super quick.

Once you've done that, you do need to restart GoAnywhere for the changes to take effect. This is going to be where you will complete the cluster process. I'll go over to my other VM here that I got running. We'll go into our config folder and we will look at the cluster XML file for this particular node. The main thing, like we showed you before, is you're going to need to come in here and define your system name, the cluster bind address, cluster bind port. Then more importantly, this clustered enabled needs to be set to true, needs to be modified from false.

Let's go ahead and actually I need to elevate my privileges here. Modify [inaudible 00:25:53] that. There we go. All right. We Roll down to this last field here. Go ahead and change this from false to true. There we go and make that change. There we can see our cluster XML file is updated. Now, go back out and the only thing that remains is to take GoAnywhere down and bring it back up and it will be ready to join the cluster. On our stop script, just to show you, we can see this MFT instance is no longer available. Now let's run our start.

Now the MFT is starting back up. Let's go ahead and monitor our other cluster here. We can see we've got, Ubuntu, support node. It is the participant role. It's cluster addressing a cluster bind port that we looked at in that XML file are available. Just note the difference here. These are different and there again the local IP address. At this point I am clustered. I'm running one thing to note, you would normally see a GoAnywhere dot log file on a single standalone instance. Once you cluster, there are nodes specific log files.

What I want to show you here is, let's go to the Ubuntu One that just joined our cluster. There it is. If we scroll down to the bottom of this, you can actually see that it is starting here kind of in its own startup. There's your startup process. It's rolling and now your GoAnywhere log, think of it as a centralized log that will track as nodes join or leave the cluster. There are these temporary files that we do for cluster validation. In that shared file system, you're going to see the little temp files that are created quickly and then deleted. This is to ensure that both nodes of the cluster have proper authority to that file system.

You are now clustered and ready to roll. That's all the process entails. Obviously, I've done it a few times a little bit quicker, but I think this is a really, really helpful document. It's a few pages long that's going to give you examples of everything and show you how to set up your clusters. There's even troubleshooting and further explanations as well. This is available out on our customer portal and yeah, very handy.

That is clustering. Real quick, did want to talk about if you did miss any of our prior Get the Most Out of GoAnywhere webinar series, here's where you can get the prior videos. They're available on demand. If any of these topics interest you, I encourage you to look at that. Also want to just thank you all for joining. If you have any questions that we can't get to in the Q&A session that we're going to roll into here, feel free to email us. My email addresses in there as well as our sales email address.

Then GoAnywhere.com has great information. You can also reach out to us via phone if that works for you. Again, thanks a lot for joining. We really appreciate you taking the time out of your day to look at this. If you have any questions and you're a current customer about the clustering process, you can email me, you can email your account manager. If you're new to GoAnywhere, then you can obtain a free 30 day trial on our website. Also check that out if you're interested as well. So hope everyone has a great day and we'll roll into the Q&A.