00:00
Greetings everybody. Welcome. Thank you for coming to Pure Accelerate as a pure employee. I'm contractually obligated to say that we're, we're deeply pleased that you're all here, which of course I am and I'm thank you for coming in uh to our segment. We're gonna talk about the controversial statement of uh the pure storage is taking over modern data protection.
00:22
The, the caveat is I can define whatever modern data protection is and therefore uh create a series of requirements that only I can fulfill. So that's what, that's what it is. But so pure storage is taking over a data protection. My name is Jason Walker. I'm the global practice leader for what we call modern data protection here.
00:41
We'll talk a little bit about that. But first my name again, there it is. There's my name again. It's the best slide in the deck and we talk about this. All right. So uh I don't know how this will play to the recorded audio.
00:54
So for those of you that are listening on demand, just know that the crowd is going wild and uh and uh every, all the bits that we'll do during this presentation in my, my interview with our esteemed guest was, was fantastic in here. All right. Uh I'll ask a question to the crowd. What is this a picture of, let's say that everybody answered
01:18
Treasure Chest, but one person in particular gets 50 points. Thank you nicely done. Um Yes, this is a treasure chest. And how many of you have ever seen a real treasure chest live? Not the stuff that you find at the amusement parks, but an actual treasure chest.
01:34
That's right. Nobody. That's right. Nobody has ever seen one. But, uh, we're told that they do exist through, uh, our, our, our cultural media, uh, movies, TV. And, and whatnot. And who often had treasure chests.
01:50
Pirates, another 50 points up here. But I do award partial credit back here at 20 points. Thank you. Keep track of the points yourself at the end. You'll find, they meant nothing. But that's ok. You, you go home with pride and that, that's right.
02:01
Pirates, pirates of that and, and pirates with this treasure, they always made it really easy to find. Right. It was always easy to find treasure chest. They just kept it on the deck, you know, that kind of thing. No, that's right. No, the answer is no, the crowd is,
02:20
uh, absolutely on top of it for the folks listening at home. That's it. They're absolutely on top of it. They're doing it. No, they, they didn't. In fact, they created what to to, to do what, what did they do to their treasure? They buried it. That's right.
02:34
And, and since they buried it, they had to find a way to get back to it. So they created treasure maps. That's right. That's it. That's it. That's our, uh, that was half of the movies in the 19 fifties and sixties, I believe. That's, that's it.
02:50
That's right. And, and, well, we did have a kind of a treasure movie in the 21st century uh National Treasure. Anybody see that fine film with The Great Nicholas Cage? That's right. And uh there was quite a convoluted treasure map associated with that, you know, and, and everything else and the whole purpose of that
03:09
was to keep the treasure safe, keep the treasure safe. Uh Now, were people looking for this treasure? Uh We're led to believe that always everybody was looking for these treasures, right? And looking for the map and looking for the clues that got them to this treasure and, and they create,
03:27
you know, some of them created a massive, uh you know, complicated treasure maps to get it to. That's how important the treasure was uh on there. Did you know in real life? Allegedly, allegedly, there's an island in Nova Scotia, Nova Scotia. That's a real place. I had to look, I had to make sure Google Maps
03:47
that it wasn't just a fictional place. Nova Scotia is a real place and that uh there's an island that supposedly has the biggest treasure out there. Like a huge golden treasure, sunken treasure out there. Now, this island has changed hands many, many, many times as owners have gone, broke, drilling for the treasure. You've heard this before?
04:11
Yeah, absolutely. It's amazing story. Uh, real people have done this. It's not, that's not, the people are looking for this, for this treasure and, you know, obviously, you know, they've never found it because they keep, they keep searching for that. Now. That's good.
04:24
That's a good treasure security plan right there. Uh Maybe too good. Uh As we'll come back to that uh in a, in a little bit, but in our world bringing it back in, you know, Nicholas cage, we'll bring it back in to the here and now in our world, you know, data is, is our treasure, data is our treasure.
04:46
And we do try to make some really creative treasure maps and systems to try to keep that safe and keep the people, the everybody else. We're led to believe that's trying to steal our data, trying to, to, to harm our data uh in some way. So in many ways, you know, the, the, the, you know, since our data is our treasure is very complicated maps and signs and we don't know if they're effective or not.
05:13
The only way that we know that they're effective is if we still have the treasure and nobody's actually successful that that island in Nova Scotia. The, the, the theory is the pirates that buried it, set booby traps that were too good and that they could never get their treasure back. A little, a little too good. So that speaks to our data security plans out
05:36
there and say, you know, what good is a data security plan if I can't get my treasure back, you know, that kind of thing. And in, in that time, and we've seen this, unfortunately, in the advent of, of, of ransomware, we've seen this where, you know, you couldn't. So some folks can't get the treasure back and some people steal the treasure and then demand
05:56
a ransom from it and then we pay it and we get 40% of our treasure back. And that's that, that's a, that's a problem. So the data, you know, data is our treasure and, and we're trying to protect it. So, you know, the, the whole title of this is pure taking over modern data protection. Well, you know, besides that bold statement, how is that?
06:16
Well, first we define data protection, right? And data protection doesn't just exist in, in a single place, in a single plan. You know, we, we create the, the Treasury Protection System. You know, we're no longer doing maps, we're doing a treasure protection system, a TPS to, to protect our data and it exists across
06:36
the different resiliency tiers. It's not just one place you might just bury your treasure you might just, or, or back up your data and then expect to get it back. That's just one way to do it. You can protect it with different, different layers of protection and different tiers and you probably should because there's gonna be some ways you can get it back faster and,
06:57
and, and then there's different levels of protection as you go along. So it's, you know, it starts with business continuity and disaster recovery. Uh taking advantage of things like replication and snapshots. There's uh you know, there's the traditional backup and recovery. It's your last line of defense, you know, the final tier.
07:13
It's like the, the, the final boss in a video game. You know, you, you get to the backup and recovery where you have, you know, you could have some snap snapshots and things of that nature and there's also stuff to consider uh from that and then you have to wrap it around the multi tier like the glasses and National Treasury, you lift one and see, see different things with,
07:33
with data security around that. How do I protect that? How do I keep it safe? What kind of things can we put real things too, not stuff that will, you know, uh that I'll never be able to get my, my data back but real, real ways uh to do that. So, and it's not just in one place, it's, you know, it's, it's different, it's different to tiers.
07:51
So out of the box, you know, pure gives you everything that you need, right, kind of right out of the box. So this is how uh pure looks kind of out of the box, how tiered backup architecture works data. That's uh you know, right from the time that the data is born, you know, little baby Yoda data, you know a little tiny data when,
08:11
when it's there, do you protect your baby when it's born? Right. Right. It's almost when it's the most vulnerable. Of course, you wanna protect it, of course, you wanna do that. So you, you know, it starts at the very beginning but then you're like, well, I can't keep it here at some point. The baby's got to grow up and leave the house.
08:26
So we, we can tier that data and send it to a second site. But still, we still love it, we still protect it. We still, we still have a ways to get there and then that, you know, you can do a third site and then you can have data bunkers and that could all be done with pure today out of the box, taking advantage of,
08:43
of some things like, you know, it's a replication and, and snaps shotting, uh etcetera. Uh But then, you know, we can all, we also have a great ecosystem of data protection partners that say, hey, we kind of do some of this, some stuff too and we work well with pure storage. So this is what it could look like with our, with our data protection partners,
09:05
adding a level of granularity orchestration uh to, to help augment what is already a great system right out of the box, right? Taking advantage of taking advantage of things, being able to send stuff to the cloud, being able to re restore things and uh and being able to uh to do that with a with a number of different things, right? A number of different things.
09:29
So it can look like that. It's an, it's an important thing to think about as you consider, how you're keeping your data treasure safe is how you can take advantage and use the most out of everybody that you're already invested with. OK, almost everybody has a data control plan, you're getting the most of that. And when you make your selection on how, which data protection software you choose,
09:51
are you considering an entire continuum of data protection? Are you considering these data resiliency tiers to do that? You know, can, can that software help with snapshot management? Can they help orchestrate? Can they o obviously, it's all hands on deck with, with,
10:09
can they protect me? R and how does it all work together? Is it simple to be able to take advantage of that? You know, you know, we think it at pure with the, you know, with the stuff that you can do right out of the box and then you know, add in that that layer of orchestration and granularity giving you multiple restore
10:29
options for your data. This works well. Uh This works well and we've seen that uh uh you know, kind of over time. So in the bus business continuity and disaster recovery, you know, you can do things with, with pure like taking snapshots and you know,
10:46
metadata, you're not cloning, you're not cloning snapshots anymore. Uh Some of some of the stuff that's in there, you can take advantage of uh asynchronous replication. You can do things like, you know, uh you know, zero RPO with active cluster near zero with active Dr uh you know, taking advantage of the uh you know, was with backup and recovery,
11:07
taking advantage of, of using that our partners to orchestrate snapshot recovery um from, from, from a data control playing standpoint. One of the most important things that folks didn't start thinking about. And when we started talking about this, when I came on board here, four years ago, 2019, we were talking about this, folks knew about the treasure,
11:29
they knew they wanted to protect it. They knew they wanted a good plan to do it and they, they would invest, they would invest a lot of resources to making this. They would, they would build three different uh reposit three different sites and, and protect it. But then when it, when it came time to, to get that data back when it came time to get that treasure.
11:49
It was so slow. Couldn't, it was like, what, what did we just do? We kept it safe. Uh But now we can't get it back. They were like that the, the Nova Scotia Island, they couldn't get that data back. So that's when uh something that like rapid restore came into play started asking the
12:07
questions. Uh It's not just about recovering a v uh virtual machine anymore. It's not just about getting an email back. Uh That, that, that seems cute. Now I go back to 2005. I have a flashback. That sounds great, you know. Uh but now no, it's about how do I restore. How long would it take me to recover?
12:27
50% of my data center. How long would it take to recover? 75%? 100%? And these are the questions our customers have begun to ask and, and you know, when we started talking about it, we would, we would review uh you know, rfps on, on things like ransomware
12:47
protection uh and recover ability. They were not even, they weren't even using recoverable again. It was really just about protecting the data and uh we weren't seeing language in there about recovery speeds and it's tough. You and your data center is when you test software, it's easy to test backup because you're just doing another copy of your production data.
13:10
It's very hard to test recovery because that recovery has to go back. Well, I could just do a VM, but that's not what we're talking about anymore. Is it, how do I test 50% of my data center? It's very hard. So, it, because it was hard to figure out how to, how to do it and how to test it. It wasn't considered as part of the plan.
13:30
But as we started to see when folks were recovering from ransomware, this was becoming a two week, three, average, 2 to 3 week to recover. Now, some of that is the chaos in the midst of an attack. But when it finally started having time to recover data, it just wasn't fast enough.
13:50
That's how pure storage got in there. I'm gonna, I'm an old backup guy. I've probably been in backup longer than I should probably admit. They probably have, you know, they probably have small circles of chairs for folks like me who have been doing backup for a long time. Tell, tell us all about your backup.
14:08
Oh, it's been 30 years. I don't know, but things have changed a lot about that and, and I was always surprised, I was surprised when peer storage said, I think we're, we're starting to see folks interested in using us in data protection. Can you come on board and help us with this? I was like, come on all flash backup and protection.
14:28
Heck, I know people still writing the tapes as I hid my tapes behind me. Uh But it was true. It's absolutely true because they were answering that question. How gosh, how can I get this all this data back quickly? And pure storage was able to help rapid restore. And that's how folks started to keep, started coming to pure.
14:48
That's when the takeover who really started to begin. Not when I got here. Although, uh I, you know, it's probably was let's be honest, but no rapid restore uh in there and being able to take advantage of that and then wrapping it around with the, with the things it, when folks call us just like anybody else in the data center,
15:07
I'm talking about ransomware. They, you know, what are you doing to help us in ransomware? Just because we're uh a, a great uh uh you know, great platform for hosting data and low latency. And just because we, we serve as a great target for, for backup and rapid restore.
15:27
Uh doesn't mean that, you know, we still can't help. And so you see uh a word up here a couple of times safe mode. So safe mode adds to the strategy. It adds to the treasure map. It adds another layer layer of, of protection and safe mode is the ability to when it's pure as a source and the data is resting there.
15:51
And we're doing snapshots to be able to protect those from uh manual eradication. They try to go in there and you try to delete it. It's like the John Candy and vacation. The Wally world puts his hand up. Sorry, folks can't do that. Safe modes on, can't, can't do that. So how do,
16:10
and then they, they try to find ways around it. But the fact is there's an au authorization gap that occurs with safe mode that you have to actually call a human and, and do that and you have to have two folks that are authorized to do it and they have to have a secret handshake and uh and uh say the secret catch phrase, the phrase that pays, they had to do all that.
16:29
And this is a tried and true method of adding to protection and it works alongside our partners and everything else. It just provides yet another hard thing for anybody that's trying to steal your treasure from trying, trying to recover there. And then we have this whole world of pure that you wouldn't think is related to data protection at all and it's operational agility.
16:51
But I'm gonna say that it's one of the most important things uh that being, being involved with pure in the Evergreen program, being able to do things with the pier one reporting adds power to your data protection plan. It, it can do things like assess, you know, what, what's out there. What am I covering something snuck in there that isn't protected?
17:13
How can I do that? How do I add safe mode simulation? How do I do some of those things in there? Uh It adds to that and then the having evergreen in place means being able to pivot as quickly as the world is determining. You have to. There's no more sad discussions that I've had uh as,
17:35
as a global practice leader for modern data protection and talking to somebody and going through and talking about the entire continuum of data protection that we can, we can provide that I covered very quickly and then look at them and they know that they can't do anything because they're stuck for two more years in a contract. Then that's something pure, pure customers
17:59
don't have to worry about anymore. And that Evergreen keeps them able to update. You heard it in the keynote, being able to continually update non disruptive upgrades just makes all of that power that we do from a data protection side even better. And it means that when things happen and you want to take advantage of something you can do it.
18:20
And that's a key operational uh agility in there just ties to our, to our other pillars uh in our, in our treasure map. All right, we're good. So you, you come up with these and I, I think we just did this to because we really like Lord of the Rings or something like that, but we did, you know the rings of protection. So now you have all these solutions that we can
18:41
provide from rapid restored to snapshot off uh snapshot offload to, you know, zero RPO with active cluster near RPO, uh RPO zero with uh active dr things like uh migrating workloads ay replication and being able to work in concert with our uh data control playing partners to bring all that together into simplicity. And then we add on all the, all the other things uh from and I haven't even talked about
19:10
the thing that I, I think was one of the more uh interesting things out of the key note, which was how much more efficient everything was. Nobody associates my industry data protection with efficiency. Yet we're able to provide that we're able to provide a, a uh you know, lower cost uh lower energy, a more efficient plan that you can
19:38
upgrade and modernize as just part of being a part of pure and you're not sacrificing any of the um of the power of performance. In fact, you're supercharging, it almost sounds like a slam duck. I had one customer. Uh tell me, ask me, he said, well, you know, why, why aren't you the target for all,
20:01
all the, all the backups? And I said that's a very good question. I'd love answers myself. But the fact is it's growing into that we're taking over and being able to, to leverage pure leverage, pure as a source uh has always been popular. But thinking about it with a data protection standpoint is, is new, being able to take advantage of multi site to do business
20:25
continuity of the active cluster to disaster recovery with active dr uh near term recovery with asynchronous replication and then be still be able to to offload the cloud and do some of the longer term uh storage things uh is is important and then protecting them immediately protect your little baby data with snapshots. OK. Being able to do that pure as a source uh is
20:49
great and then as a target is great too, taking advantage of this is part of your data protection plan. Now, we have you heard it, you heard it on the stage, have yet another one where we started with rapid restore and it all started with the flat with the original flash blade, what I call flat classic flash blade, anybody, any peer customers in here that have uh one of the first gen flash blades,
21:12
I call that classic flash blade. I don't call them first gen it's not a robot sitting in the corner, you know, not a T 200 or something like that classic flashlight. It started with that and, and the response in the market was so great that it very quickly became a part of, of everything that we do at pure, that rapid restore, being able to recover quickly, super charging some of those data
21:36
control planes, you know, the the those partners that I put on the board partners like and they all have done something specific in the last 10 years. And cos too, they've put a ton of great function in their software and that's where they are software. They've done a great function in their software. There's only one problem for them out there is
21:58
that the targets that their customers are choosing for, that aren't performing enough to make, to help them realize that function, they slow read, they can't do any of the the great data security things. I wanna scan that before I bring it back. Well, that requires you to read. And if you're reading it off one of those legacy hard drives that Charlie and Sean talked
22:20
about, it's gonna be slow. So it, it take it supercharges everything. And so, you know, little by little in, in every new platform. The question is, you know, how do we, you know, how is our customer gonna take advantage of this for data protection?
22:38
And the answer is very, very easily. No super super charge. What's currently existing? The number one thing I know from being in backup a long time is that nobody wants to change your backup software. It's hideous, hideous.
22:53
And uh so can you come, can you help us pure storage? Can you help us come in and supercharge what we have? And the answer is absolutely, let's talk about how to do it. What makes the most sense? Not just in one site, but we can do multiple sites and you know, some of the Softwares can even track our snapshots
23:13
in an active cluster pod and to a third site off the back of that impressive stuff. And that's the kind of thing you're thinking about as you're building your, as you, as you're writing your new treasure maps out as you're working on how to protect that, that treasure chest uh of data there. So yeah, it's, it's a party and all the platforms are invited to it.
23:35
Great response uh on it. And just a matter of what fits for you, the outcomes you're looking for. How fast do, do you need to recover it? How fast do you need that? 75% of your data center back? What kind of things are, are, are you looking at at protecting?
23:50
What kind of things are you doing? How can we help? We often will get calls about one item. I know nobody's taking a picture of me. So I'll make sure I stand off to the side. Uh and that, and for the people listening on demand, that's a visual bit right there. I'm way off to the side.
24:09
Um The what we get called to supercharge as a target and then, you know, don't be, don't be alarmed when we ask you. Well, what are you doing at the source? What is your tiered plan? It's not just A to B it could be, but isn't there something that you can do with snapshots
24:27
today? I mean, don't you? That's the fastest way to recover as much as a backup guy. I am snapshots the fastest way. What can we do? Because it's so easy to use with pure and then we can protect it with safe mode. Beautiful, beautiful, so great, great targets. Protect it with safe mode if I could
24:46
leave you with anything. Um from a, from a uh an old one, old pirate to everybody else. Is that be intentional about your plans? Make it multi tiered to make it easier for your business to recover quickly. Always consider restore as the most important part of the entire plan.
25:12
You could do all the other stuff you could have, like I said, three sites and you can have uh smart draw bridges that come down and you could build a giant moat all around. But if you can't get your data back in a reasonable amount of time, you'll probably quit before you even try to recover in a, in the case of a disaster.
25:31
You won't put yourself in that position. Let us put you in a better one uh with that. So, uh that's it. Now, you, you heard me monologue for about a half hour, but I think it would be uh better to hear from somebody who's been in, been in the industry.
25:48
Uh Quite a long time. I don't want to age you that much. Uh a new relative newcomer into the business. No. Have a have one of our customers come up and talk about, you know, the data protection uh experience and having gone through a couple of things himself. So please uh help me welcome,
26:06
you know, uh Sean Hallahan, who's the vice president of it, Infrastructure and Operations at a FN I corporated there, Sean. Welcome. Thanks. And those applause are largely because soon I'll stop talking. There you go. There you go.
26:24
So, uh Sean's mic is up. Go ahead and hold it. I think you have to hold it closer. I know it looks like I'm eating ice cream. I gonna do this. But I think that's, that's the way it is. So Sean, tell us about, you know, your background, uh tell us about the company and, and then your, you know, your background.
26:40
Uh Yeah, is a company that no one's ever heard of, but most people have interacted with. Uh we run call centers and chat centers for largely for the telecommunications and insurance industries. So large footprint in the US, Mexico and Philippines, right around 15,000. Call center agents. Wow, 15,000 call, call center too.
27:03
Call center, call center. Al always uh always up. You don't wanna just give, always uh 8000 people some time off when there's down time. That is correct. Ok. I just wanna make sure, you know, that's my assumption but, you know, I, I do assume we're wrong from time to time. Um You know, and you tell us about your own background in,
27:21
in, in, uh, it and, and protection as well been in it for 20 some odd years. Um, kind of around 15 years ago. Kind of went storage as my specialty. Um, so was a backup guy as well. Yeah. So, hold on that. That's a key point right there, Sean's bringing up back up,
27:42
you know, not many people stay in, back up. Uh, I do all this stuff. I didn't. That's right. That's right. So that's another thing too. You wanna build your treasure map so anybody can use it because typically we rotate people through data protection and we want to make these things.
27:58
Uh Well, you say it, I'm, I, I, I'm from Daytona Beach, Florida. OK. So you have a real life Florida man standing here uh in here. And the, the biggest thing we help customers with Sean. I don't know, you know, we talk about, we talk a lot about ransomware and pirates and, and coming in and certainly you have that and
28:14
compromise a credentials, what I would call the Oceans 11 type methodology where, you know, look over here and they're, you know, taking something off your belt, right? Uh But the most common place we help is uh protecting against Florida man. A the accidental admin uh on there and safe mode could help there as well.
28:32
But go ahead. Uh So 11 years ago came to a, as a what, what at the time was called a server engineer. But uh we really didn't have dedicated storage people. So I was kind of the storage by default. Um, at that time they were and uh pre H pe three part
28:57
shop, um, original three part classic. Yeah. So, um, that was kind of my background, um, moved up, start when went into management shortly thereafter and have kind of gone up since then. Absolutely. Well, congratulations on, on doing that and not getting stuck in backup thing uh in there. But we've seen things change a lot though.
29:19
I mean, over, over the years, over the 20 some years, we've seen data protection change a little bit. Yeah, like, so what are some of the ways you've seen from your experience? So, you know, a he didn't have a great data protection strategy. Um You know, it was something like this.
29:41
It, it was more of a check box on a compliance form rather than, rather than a strategy um, that changed for us. Uh June 7th of 2021 at 11:57 a.m. The time stamp is the time stamp is, yeah. Um What happened then? That is when we got hit by ransomware. That's when it went active.
30:07
We had 550 servers and 8000 end points go down in about a three minute period. Oh, my gosh. Was I right. Is there chaos when stuff like that happens right off the bat? Yes. Um The hardest part was getting someone to make a decision,
30:26
right. Let's turn everything off. Let's, we have to turn everything off, guys, we need to turn everything off. Um You know, I, I can remember being on that bridge saying that um at that point in time I was director of infrastructure. Um wasn't sure if I was able to make that call.
30:45
Uh I wound up making that call had to be done. Um You know, no one and no one was in the office, everyone was remote. Um And it was, it was a horrible situation. Uh Since we're all friends here at the time, I was barely a pure customer. Uh We had bought one X 30.
31:07
Um It qualifies to be a logo on a screen but you, you're definitely correct dipping your toe in the water and it was not in production. Wow. Ok. Um So we were in the process when COVID hit, we had about 80 remote workers. We went to right around 8000 remote workers. Um We cratered our sand when we tried to uh move all those people to VD.
31:34
I uh had tons of issues because we just increased capacity and it was an all flash three part at the time. Um So you're already doing all flash, but it was three part, I'm doing all flash, but it's three part. Ok. Um So, you know, worked with a bar and was getting ready to do a bake off.
31:50
Um So we had 13 part that had been delivered to a Colo, but it was a brand new Colo and we were still waiting on core networking. So we had a box sitting there but doing absolutely nothing. Um When ransomware hit, uh our first step was let's go to backups. Um At that point, we were customers.
32:13
Ok. Um We went to do restores from our local bricks. We restored them, powered them on. They'd hit our NTP server and immediately began encrypting the ransomware, sat a little over 10 days on our systems before they activated it. So to keep costs down, uh we shot for 10 days or seven days on prem and then
32:41
we moved everything to the cloud. So we didn't talk beforehand. But when he talked about the importance of restore, I think we restored less than 10% of our servers because of the time to bring them back. We rebuilt uh over the course of three weeks. So we were, we were hard down for seven days.
33:04
Um Nobody doing anything and we were three weeks to recover fully. And I say fully. Um we, we measure in our industry, what's called tech time. So it's the amount of time an agent isn't doing their job for technical reasons. Uh We typically average around 5% agents look for reasons not to work,
33:28
right? Um We typically average around 5%. That's what we budget for. Um between July when we were back up and running in November of that year, we average about 12% but everything in our environment was rebuilt new by people working 18 or 19 hours a day. Not, not at their peaks.
33:49
So you went in, you didn't really sounded like there wasn't a real action plan when things hit, there wasn't, you had to grab the mic and say, turn it off and then you, you, you found the pain of her store. Uh That's a real story and that's, that's about right. Three weeks of, of that and it wasn't and a lot of that was manual rebuild.
34:08
It wasn't even rest restoring place. One other one other thing that, that came up um during that period. So Tuesday, we got hit Friday, I get word from our insurance company. I can't delete any of my encrypted V MS. So I suddenly need double the storage space. I currently have um We run our
34:32
sands at 70 to 80% capacity. I mean, we, our, our margins are relatively thin. We don't have extra hardware sitting around. Um on a Friday. I called every storage vendor and every bar I land on this chair that you rubbed and then the,
34:51
the storage genie just kind of came out and said, I'll grant your wish. It was in the middle of COVID. I mean, you couldn't get equipment. Um Definitely. So I called everybody I could think of um one of our storage people. One of presidio was the, the, one of the bars I was dealing with,
35:10
reached out to the pure rep that I had bought one X 30 from, um, on a Saturday or Sunday, Sunday. Dave standing in the back. Dave. Dave youth is my, my pure rep. He remembers it. Well, 9 p.m. these time stamps are awesome. Yes. So I'm, I'm in Bloomington,
35:29
Illinois 2.5 hours from Dave. Um Dave happened to have a Poc array that he, I think he was gonna give another customer and he somewhat hijacked that and had it in my data center around one o'clock on Monday afternoon. And by three o'clock we were moving V MS over to it and that's great. Um And then, yeah, and then Wednesday, another pr showed up in my data
35:57
center in Phoenix. Um, we ran on those for seven months until they released it. Since then, I've bought multiple more flash arrays, uh flash blades. Um So coming out of this ransomware that where we really were as executive committee came and said this took us three weeks. We need to shut down to three days,
36:20
come up with a plan, three days. We want full restoration. So one of the things you guys learned coming out of that was unrelated, although you learned some stuff about getting more pure. We, we're always in favor of that was let's have a better plan. Yeah, I mean, it in response but response and everything else you had three culturally,
36:38
we were very much, we achieve security through compliance, going into this event. Um, coming out of this event, we achieve compliance through security there. You know, it's, it's, it's a different mindset. Um, you know, everything was ok. We need to check this box. You know, I actually, we actually made it a company policy that in a change control for a
37:02
reason you're doing it. It can't be PC I, you know, or it can't be or you know, it has to be a legitimate security reason why you're doing it. Um Not that those aren't, but that couldn't be your reason. So, so yeah, so I mean, you come out of it, you got a new mandate to, to do a three day plan. But why even do that,
37:23
Sean? Don't we know that ransomware is like chickenpox, you don't get it twice, you don't get it twice, right? I mean, no, no, you, you were wise to continue on. So we went with um yeah, flash recover uh with cos so we're backing up directly to flash blades.
37:42
Um If you don't know the flash recovers our combined solution with our alliance partner cos where has specific compute nodes, no compute only and dis aggregates and makes flash blade to take advantage of that rapid restore, to restore faster, even even faster there. Yeah, so we also put into place every week, we uh recover 100 and 50 V MS um into a non rout V
38:10
land that does have access into our test Aws and Azure environment. And our Q A teams go in and do full uh quality testing on 100 and 40. So you restore 100 and 50 roughly between 100 and 40 100 and 50. That's a week. That's a great point that I don't hear from a lot of customers and full application testing.
38:32
Yeah, I mean, to add restore testing into your hygiene to your regular cycle and it's critical again, it's tougher to do restores than it is backups. But it, it sounds like it was important to, to take, to take that. Yeah, I mean, we have that, you know, it, it, the, the budget for backups became much larger after ransomware than they were.
38:55
I bet three weeks down we'll do that to you. Yeah. Yeah. Um So, yeah. Um, you know, that that was a big part of it. Um, in on January 3rd of this year. Um, we had a malicious actor, uh, get into our environment.
39:15
Uh They called into our help desk claimed to be one of my engineers. Oh, wow. Um, or no, sorry. They, they called first on New Year's Day, so January 1, everyone's out. Um The shields are down mentally called into our help desk in the Philippines because there
39:34
was no us staff on at that time. Um said I, you know, I can't log in. I need you to reset my password. They reset his password said, yeah, I still can't get in. I think there might be a problem with my M fa can you remove my UB key from my account? They did the help desk person then says, I notice you also have a Microsoft authenticator.
40:00
Do you need me to remove that too? Oh, wow. They removed authenticator. Um, they got in created 17 accounts and didn't do anything, right. It's blueprint time, right? The third. Um, so I think a Wednesday, um I've got,
40:22
it's seven o'clock in the evening. Um, I'm at do Wednesday is my D and D night. Um, yeah, my, my network manager, my server manager, a couple P MS and a security guy play in my DND sessions. Ok. So we're all sitting around the table and all of our phones go off and they're thinking it's
40:43
part of something you created. You know, this is all part of the, uh, simulation, right? All of all of our phones go off. We're getting signal alerts, um, from our sock. Um, someone has tried to turn EDR off on multiple servers. Um, we get on pretty quickly, um, identify those accounts fairly quickly.
41:06
Um Basically any account that was created between Jan One and Jan th and June or January 3rd, we destroy those accounts. Um, and then it becomes them creating accounts and us destroying accounts and, and we play that game um it turns out they got into three different servers. Um We were able to shut those three servers down and power them back,
41:33
shut them down. You know, we're in full production at this point. They are production servers. Um Most of them are file servers in the 18 terabyte range. So large windows file servers. Um We shut those down, power them up while they're still sitting on the flash blade.
41:49
Um Our total downtime is right around three minutes. Wow. Um, so it's VM recovery mass restorer in the and uh able to stand those up while they're VM ware. So they run on the flash blade, able to do that at scale too. That's what we're talking. Yeah, when we stand up those 150 you know, restore those 150 machines,
42:12
they never hit our production sand. We restore those up on the flash blades, do full Q A testing and everything. Um So is in the, the restore is literally the time it takes the VM to boot. That's, it's incredible. And that's not what you get on the legacy stuff too. There's no fast read for that.
42:27
Yeah, there's no rehydration. You, you, you're booting a VM uh the VM boots and then it does storage migrations back to your primary storage in the background. Incredible. Um You know, while, while we were all sitting there, um, I happen to have an Apple watch on at the time and I started getting notifications, you know, your, your heartbeat is above 100 and 20 beats per minute.
42:48
You don't appear to be exercising. Um, quiet apple. Right. You know, so I know I'm stressed. Right. So, you know, it's, it's when they talked earlier today about, you know, getting your, your time back and, you know, being able to give your people nights and weekends again,
43:06
you know, this was probably as invasive of an attack as we had in June and it was three hours and we were out of it and realistically that was just clean up. I mean, while we were working, it was less than an hour. Amazing, amazing stuff. Thanks Sean. You know, one of the other things I'm gonna bring up real quick.
43:29
Um when we had our ransom, we, our cyber insurance um after post attack, after we went in and put things in. Um, you know, and pure was a big part of that and the flash recover is a big part of that. It actually lowered. Our cyber insurance went down by 33%. Wow, hosted 33% insurance company lowering uh after an incident.
43:55
That's truly, that's a miracle. I mean, it, it's, it's been, it was big for us. That's incredible. That's a good, that's a good tidbit. That's, that's great. Absolutely right, Sean Haa. That's a great story. We have to wrap it up as you heard the applause
44:11
going on all around. That means we have to wrap it up. David culture of. Right. OK. It's awesome. Yeah. So for those on the recording who they can't possibly hear David from way that far, uh There's been an announcement for a ransomware
44:45
sl A is a part of our evergreen one program. Definitely check into to that and we're able to do that for some of the same reasons that, that Sean just shared with us. Here's some other ones, check it out. If you're looking at the slide online, uh you've already missed these, go listen to them online. And if you want more shenanigans with me, I'll be at super charge and secure your MD P board
45:04
closes at flashlight uh tomorrow, but definitely check them all out. Yes, and a flash talk. There you go. If you can't get enough data protection, we've got it for you. We'll have it in there. All right. Well, thanks for, thanks for coming, everybody. Enjoy your time uh with pure accelerate and uh
45:26
we appreciate it. Thank you again, Sean for coming up.
