So even amidst a surging global pandemic,
machine learning in seismology is difficult to avoid.
So I’m going to use the term “machine learning” and “deep learning”
in this talk to mean a data analysis technique that involves taking a model,
or a parameterized functional form, and optimizing the parameters
in that model to produce some output, given an input.
And so the reason that people are so excited about these new data analysis
techniques is that they are both very efficient, because once trained,
they’re essentially analytic, and also they’re high-performing.
In many cases, machine learning or deep learning techniques can exceed
the performance of humans when they’re given the same sort of task.
And so this enables the sort of bulk analysis of tremendous
quantities of data. And, as we all know, seismology is a very data-rich field.
This is, on the right, the size of the IRIS DMC archive as a function of time.
And, as of this month, it’s approaching around 800 tebibytes, so just a huge
quantity of data that we want to analyze, right?
So a number of ML algorithms have been developed in geophysics
and seismology. They tackle everything from earthquake early warning
to ground motion prediction. But what we’re going to focus on
on this talk is basically using deep learning
for earthquake detection and phase picking.
So a typical deep learning approach to seismic phase picking would involve
the following steps. So first you need to assemble a training data set,
much like the one that’s on the right. And that would contain basically
three-component seismic data and the associated P and S wave
arrival times for each one of these records.
Next you want to set aside some small portion of the data, which is called
the testing data, and we use this later to assess model performance.
And then you use the lion’s share of the data for what we call training,
which is this parameter optimization process.
We define a model architecture. In this particular situation,
we use this thing which is called UNET, which is a particular type of
convolutional neural network. We selected this just because it’s
been shown in previous studies to work really well
for this particular application.
We define our model inputs, which are these three-component seismograms,
and we also define model outputs. The model output is sort of something
that you have the freedom to select, but again, with the previous studies,
they’ve found that using a Gaussian functional form, where the Gaussian
is centered on the analyst’s P wave arrival pick and the analyst’s S wave
arrival pick, works really well. And so what we’re trying to do here
is develop a tool where we can just basically take and apply it
to continuous data streams. And anytime it thinks that there’s
a P or S wave arrival, the output will be sort of a peak, okay?
And the amplitude of that peak is a measure of how confident
the network is that there’s an arrival there.
Next, we train the model, and then we need to measure how well it performs.
And, for the purposes of this talk, we’re going to use two different metrics.
So the first is relevant to the classification problem.
So just, if you give the network a particular window of continuous
seismic data, how well can it tell you whether there’s an earthquake
in that window or not? And, in particular, we’re going to
focus on just the accuracy, which is maybe the most simple metric.
It’s just a measure of, how often did the network get it right.
Whether it said it was an earthquake or wasn’t an earthquake, when was it right.
The next thing we’re going to focus on are picking metrics.
And so, for the picking metrics, what you typically do is you take where
the network thought the P wave arrival was, and you compare that to where the
analyst thought the P wave arrival was. And you can make – you do this for
many different examples, and you can make histograms of time differences
between the network and the analyst, and that gives you some estimate
of how well the network is doing.
Okay, so that’s deep learning and phase picking in a nutshell.
But, as I said at the beginning of the talk, you know, the promise
of deep learning – and it’s just a tool, but the reason that we want to use it is it,
you know, paves the way, maybe, for a next generation of really accurate
earthquake catalogs. And these catalogs can be assembled really quickly, okay?
And so one of the places that we’re data mining, i.e., trying to find
as many earthquakes as possible, is in Cascadia.
But you have to walk before you run, and thinking about Cascadia, it’s hard
to ignore the fact that a tremendous amount of the seismic data in Cascadia
is kind of non-traditional seismicity. So, in particular, what I’m talking about
are low-frequency earthquakes. So Cascadia is a type locality for
slow earthquakes, which many of you may know.
The Cascadia subduction zone has slow earthquakes approximately
every year in different parts of the margin. They occur down-dip of
the sort of locked section of the megathrust in Cascadia.
And these slow-slip events are accompanied by a very weak
seismic signature know as non-volcanic or tectonic tremor.
And so, if you – if a slow earthquake were going on, and you were to,
you know, grab the seismic data across a network, what you would
see is something like what’s up here on the right.
So this is seismic data from many different stations in Oregon
during a slow earthquake that happened in 2011.
And so you can take, and you can filter these traces,
and plot them up in this way. And what you can see is, A, if you were
to compare this to a time when there was no slow earthquake going on, there’s
far more energy in this frequency band than there otherwise would be.
But also you can see these sort of coherent amplitude changes
that tend to manifest across the network.
And if you scrutinize these further, you can see that they have
a particular move-out. And scrutinizing them even further,
you can tell that some fraction of this tremor signal is made up by
repeating seismic events known as low-frequency earthquakes.
And so low-frequency earthquakes, they look a lot like typical
small-magnitude earthquakes, so you have things like P and S wave arrivals,
which you can see in the trace here. But they’re a little bit different,
just because they tend to be really, really low amplitude.
So signal-to-noise ratios are around 1 in many cases.
And they also are depleted in high-frequency content relative to
traditional earthquakes of the same size. And so I’m going to show you
something that I thought was kind of interesting.
So what we did was we took our – you know, our earthquake data set
with the selected P and S waves. We trained a network just like PhaseNet,
which is Zhu and Beroza’s convolutional neural network that
was trained on data from California. And then what we did was just, like,
apply it to a bunch of different examples of known earthquakes in the catalog.
And that’s what I’m going to show you on the right-hand side.
So on the top are the three traces – Channel 1, 2, and 3.
Sometimes they’re identical because, if it’s only a vertical-component station,
we copy the vertical to the other two components.
So this is the input. And then down here is the output.
And so what you can see, hopefully, is that the network seems to be
doing pretty good. You know, when you have clear P wave arrivals,
you see a purple Gaussian that’s very near 1 in amplitude.
And, when you have clear S wave arrivals, you see a purple Gaussian
that’s very near 1 in amplitude.
And so this seems to work really, really great. Wonderful.
One thing that we noticed [chuckles] though is, if we take that network and
apply it to low-frequency earthquakes, it definitely does not do great.
So, just to guide your eye, I’m showing you a stack of LFEs up here.
So this does not change. So this is the Channel 1 stack,
the Channel 2 stack, and the Channel 3 stack.
And this is just to show you that there is P wave – there is a P wave here,
and there is an S wave here. Down here, though, are the individual
traces that go into making up the stack, and this is what actually goes
into the network. And, when we dump these into the network,
there actually is a prediction down here. You can hopefully see that
occasionally they’re non-zero. But, by and large, the network is saying,
this is mostly a zero, guys. Like, this is noise. It’s not signal.
Okay. [laughs]
But we know that there’s signal there, right?
So what we want to do, or what we want to ask is, like, can we use the
same deep learning approach that we’ve already talked about that’s been
applied to traditional earthquakes and tune it to detect, or train it to detect,
low-frequency earthquakes? And it may seem obvious that
you can do this, but it’s definitely not obvious that it’ll work.
And the reason is just because LFEs are really low-amplitude.
I know I’ve said that before. But each one of these light gray traces
here – this panel is Channel 1, this is Channel 2, and this is
Channel 3 – you may not believe me, but each one actually contains an LFE.
It’s just they’re so low-amplitude, you just can’t see it in the raw data.
What you have to do is take and stack these guys.
And ultimately, in the stack, you will start to see a P wave arrival
and an S wave arrival pop out. So what we’re asking is, can we put
this into a neural network and have it give us something meaningful?
So the first place that we tried this is Parkfield, just because it has a really
great LFE catalog that I’ve worked with a lot, and it was assembled
by David Shelly in 2017. It has over a million LFEs in it.
And so what we did is just follow that procedure that I talked about at
the beginning of the talk. We put in three-component seismic data.
We asked the network to predict when the LFE S wave arrival is.
And the only really modification that we made is that we started expanding
the width of the Gaussian that we used just to give it a little bit
more allowance for their – for LFE arrivals being emergent
and low-amplitude. So, okay, so that’s that. [laughs]
Okay. So we did this. We trained it.
Again, things are really low-amplitude. It’s really hard to see just individual
LFEs in the continuous seismic data. And this is the result in a nutshell.
The peak accuracy is about 86%, meaning, if you take this network
and use it as a classification problem, so you give it some window and you
say, is there an LFE in this window or not, it gets it right 86% of the time.
And that is pretty impressive considering that PhaseNet,
which is one of the many deep learning pickers, but it’s trained on earthquakes,
its peak accuracy is, I think, about 95%. And so we’re not –
we’re doing worse, but we’re not doing that much worse. [laughs]
You can also compare the picking performance of the two.
And so that’s what I’ve got here. PhaseNet was trained on
100-hertz seismic data. So this is minus 20 samples to 20 samples.
If you compare the kind of same interval on this plot down here, minus 20 to 20,
you can see that the distribution is, like, not quite as sharp, but it still is peaked.
And so it’s definitely getting in the neighborhood,
but just because probably of the low amplitude and the emergent
nature of LFEs, it’s just not quite doing as well.
So you can take – after you have these trained networks, and you can
go and you can apply them to continuous seismic data.
I’ve got one example here for you. These are – each row here, each panel,
is a different seismic station over some short time period.
The black diamonds mark times of – arrival times of known LFEs,
so LFEs from the Shelly et al. catalog – or, just the Shelly catalog.
And the circles are when the network thinks that there could be an LFE.
And what’s really neat about this is you can see that the network seems to
be identifying low-frequency earthquakes that are in the catalog,
but it also appears to be identifying a lot of things that actually do
look like LFEs. So they have the right frequency content.
They get detected, in many cases, sort of across the network.
And, if you take and you go the next step, and you actually template-match
them, you can get a P wave to emerge. So this network is detecting known,
but also what I think are new, LFEs in Parkfield.
And so we’re continuing to explore this today.
So, just as a very last note here, we did the same thing.
So we trained a different network to detect LFEs in Cascadia using
Michael Bostock’s catalog from southern Vancouver Island.
So this is Vancouver Island. Each circle is an LFE epicenter,
and they’re color-coded by the depth. And then the triangles here are the
network that Michael Bostock used to detect LFEs.
So what we do is we go through the same training process
as we did in Parkfield. And we’ve gotten a little bit further
along in Cascadia because we’ve assessed network performance, but we’re actually
starting to do the data mining, where we just take and apply
this network to chunks of continuous seismic data and
sort of scan through each day of data.
And then we have catalog detections that we can explore.
And so this is a figure from Michel et al. where they did slip inversions for
slow earthquakes in Cascadia. And I just want to – I just want to
bring your attention to this box right here – this red box.
This is essentially like the latitude band that southern Vancouver Island is in.
And so we’re going to use this in the next plot.
So, when we take and we mine through continuous seismic data,
we see some pretty exciting things. So first of all, each row here is
a different station. Time is on the X axis,
and we’re just plotting the number of detections per day, and we’re comparing
that to Michael Bostock’s template-matched catalog.
And what you see is that there are near-simultaneous peaks across
the network in many cases. And, if we go back to those Michel
results, these near-simultaneous peaks line up with known slow-slip events.
So this thing is detecting LFEs on a grand scale.
And it’s doing exactly what we hoped it would do, which is detect them during
times of known slow-slip events. But also, if you look at these,
they’re really rich in character. There’s really a lot here that
should be explored. It looks like there may be smaller slow-slip
events that generate LFEs and so on and so forth.
So this is just a first cut, and it all looks very promising.
I don’t have that much time left, but just briefly, like, deep learning does
seem like it can successfully identify LFEs in a way that is useful for us.
It’s capable of identifying new and known LFEs.
And early tests in taking and applying these networks to continuous seismic
data yield really promising results, i.e., we see lots of LFE detections
at times we know we should see lots of LFE detections.
And then there’s even more rich stuff in the catalog that we haven’t yet explored.
It’s probably best to go ahead and use an approach that combines
these deep neural networks with template-matching.
And we can talk about that in the discussion section after this set of talks.
But I just wanted to leave you over here with – if you remember the
dismal performance of the CNN picker trained on earthquakes on LFEs,
this is our LFE-trained picker looking at LFEs, and you can see that it seems
to be doing a little bit better. So thank you for your attention.