Praat notes

From Helpful
Jump to navigation Jump to search

Language units large and small

Marked forms of words - Inflection, Derivation, Declension, Conjugation · Diminutive, Augmentative

Groups and categories and properties of words - Syntactic and lexical categories · Grammatical cases · Correlatives · Expletives · Adjuncts

Words and meaning - Morphology · Lexicology · Semiotics · Onomasiology · Figures of speech, expressions, phraseology, etc. · Word similarity · Ambiguity · Modality ·

Segment function, interaction, reference - Clitics · Apposition· Parataxis, Hypotaxis· Attributive· Binding · Coordinations · Word and concept reference

Sentence structure and style - Agreement · Ellipsis· Hedging

Phonology - Articulation · Formants· Prosody · Sound change · Intonation, stress, focus · Diphones · Intervocalic · Glottal stop · Vowel_diagrams · Elision · Ablaut_and_umlaut · Phonics

Analyses, models, processing, software - Minimal pairs · Concordances · Linguistics software · Some_relatively_basic_text_processing · Word embeddings · Semantic similarity ·· Speech processing · Praat notes

Unsorted - Contextualism · · Text summarization · Accent, Dialect, Language · Pidgin, Creole · Natural language typology · Writing_systems · Typography, orthography · Digraphs, ligatures, dipthongs · More linguistic terms and descriptions · Phonetic scripts


Everyday Praat

How Praat thinks: The list

Things are given an entry in the list when you

  • record sound
  • load files
  • make list items based on other list items

You can also save each item in the list to a file.


Loading and saving have to be done explicitly - the list is a temporary scratch space, not a project.

You would be forgiven to think this is a project, where you can save everything you see.

It is not - you can not save the list itself, and it will not be remembered between runs of Praat.

The list is itself intended as a temporary scratch area, "the things I need for what I am currently working on", the intermediate steps you need to do a nontrivial task.

Which means that, whether you are working interactively or writing scripts, you have to keep track of what you left in the list, by name or ID.


You can select one or more objects from the list, to then use one of the (applicable) buttons, to do something useful with this object, or combination of objects.

Aside from doing that by hand, you can do the same by code (by name or by id), which makes it easier to create scripts (but more on scripting later).


Notes:

  • (We lie: technically you can save all objects in the list, and later restore that, but this is quite inefficient, and you probably do not want to if you do not need it)



Objects and object combinations

For many objects, when you select just that object, you will get a View & Edit button.

That button would open a window for it to, well, view and possibly edit it.

e.g. try View & Edit on a Sound


In some cases, specific combinations of objects ...

  • ...brings up some specific new buttons
  • ...hides buttons that do not apply to this combination

For example, select a Sound and a TextGrid.

You will now get a View & Edit that is different: it combines the sound view and the textgrid view -- meant for annotating sounds
...and yes, it allows you to do that for any Sound-Textgrid combination, but it usually only makes sense when you created such a TextGrid from the Sound:
select the Sound, then click AnnotateTo Textgrid...


Notes:

  • Sometimes nothing makes sense for the combination you have selected
  • There are also hints -- e.g. when you select only a TextGrid, there is a button named View & Edit with Sound? which, if clicked, just tells you to select both.
there are many possible combinations not hinted at -- just some of the most common ones

Object types

There are quite a few object types.

Many of which you may never use.

You do not need to know this yes -- you'll learn them as you use them -- but roughly in order of how quickly you will probably see or need it:


Sound object, View and Edit window

Sound[1]

waveform (PCM data)
often mono, but can be multi-channel
View & Edit shows waveform and spectrogram
as the legend hints:
blue trace is pitch (note it's on a different Y axes from the shown spectrogram, because it'd otherwise be at the bottom)
red dots are formant places
green/yellow is intensity
pulses are shown in the waveform

There is also a LongSound[2], for things that won't necessarily fit in memory, but this is less of a concern these days in terms of RAM


TextGrid, here with only interval tiers
Sound and TextGrid together makes a lot of sense while manually annotating (select both, then View & Edit)

TextGrid[3]

lets you annotate text
...usually shown together with the Sound it is annotating
either annotating text with
a time range - start and end time
a point in time



Strings[4]

ordered list of strings

Table[5]

Matrix[6]




Pitch

Pitch[7]

periodicity candidates over time, with relative certainty
in regularly spaced frames - even though the evidence used for each frame are not, so may come from different amount of pulses. (Pitch shows it regular, PitchTiers does not(verify))
may be analyzed separately, from Sound objects; there are varied methods (the one shown here is intentionally a messier variant)


PitchTier

PitchTier[8]

basically a set of (timestamp, pitch_in_hz)
probably extracted via a Manipulation object
Praat itself, if asked about pitch at a point, interpolates between these points (and extends outwards before the first and after the last value)
can also be altered (and drawn) to resynthesize LPC/PSOLA type things with different vocal pitch



Manipulation

Manipulation[9]

LPC/PSOLA style speech analysis of Sound object (a specific model which happens to e.g. separate pitch, duration)
View & Edit shows
pulses (if you extract this, it becomes a PointProcess)
estimated pitch (extracted as PitchTier)
duration (extracted as DurationTier)
contains a little more
in particular the original Sound, e.g. for comparison's sake


The praat picture window

Picture window

Mainly used for making plots from data.

...which can then be saved as raster (PNG) or vector (EPS, PDF).

You won't need this until you do, so can close it.

  • ...I've seen people making startup scripts to specifically close it
  • You can also start Praat with --hide-picture


Automation and scripting

As hinted at above, you can automate Praat.


It has a scripting language, which has

  • assignments and expressions and loops and such, as most programming languages
  • the ability to call registered actions -- basically, whatever is a button can also be done from a script, and it is equivalent to doing that yourself.


In fact, aside from finding out what that action is called and typing that into a script, these is an easier way to start making scripts: because (almost) everything you can do in Praat GUI is recorded into history[10], you can at any time do PraatNew Praat Script, then in that new window EditPaste history.

You will probably have to remove a lot of irrelevant actions, yet is a simpler start than writing it yourself.

So no programming required, for some of the simpler automating of sequences.

💤 (This recordability of actions is also part of why Praat seems a little clunky at first. There is no fundamental difference between functionality already in Praat, and what you add later - it's all just buttons triggering specific actions. This action nature is also why some actions have old (and sometimes stupid) defaults: to not break older scripts relying on old defaults)


That said, learning how to program in Praat does make your scripts more flexible. It adds, among other things,

  • a way of ask for user input (primarily for the parameters you then hand into an existing action)
  • a basic scripting language that lets you do conditions and loops and other basics you will probably need to express your wishes.

To get into actual scripting, see Praat scripting notes.

Common tasks

Recording sound

Usually: NewRecord Mono Sound

You can record one or more fragments, and Save it to the List.


💤 Why Mono?

Most microphones are mono, and a lot of sound analysis works on mono.

Stereo has directional effects when you listen to both at once. This absolutely has its uses -- in particular, if you are recording less for sound analysis, and more for a person to transcribe it to text later, know that a lifetime of having two ears makes people better at separating multiple voices from any sound recorded in a vaguely spatial way, than from mono. This is part of why portable recorders tend to have two mics mounted on them.

Also, it is good science to only vary the things you intend to study, and stereo effects can boost or suppress frequencies, and will sound different in speaker versus headphones, and other such details. This is easily overstated, as the effects are usually just a few decibels, but mono will make your life slightly simpler, so is generally preferred.


Recording interfaces tend to have two input channels. What happens when you ask for mono?

Turns out this depends on the recording software and on the device. You may just get the first/left, you may get a mix of both. Be sure to turn down any channels you are not using, to avoid its noise.



Keep in mind, however, that if you have connected multiple input



On levels, noise, and scaling

🛈 To set up reasonable recording levels, before you hit record (and note this should be similar to any "gain staging in 5 minutes" intro)

Hit record in Praat, tell your subject to talk at reasonably loud levels, and look at the green/yellow/red indicator

  • if
it doesnt move, increase the input/mic sensitivity until it does
if it barely noves, probably still increase it a little more
it frequently gets into yellow, probably also lower it a bit
it goes into the red, certainly lower it because it's going to distort
for a quick setup, the aim is to get it to be middling, not to get it to be the loudest (in that this way you're leaving space for people to be a little louder than planned, without it necessarily distorting)

This puts levels into the right order, but isn't perfect advice - hardware varies, and there may be other reasons for e.g. things to still be quiet, or for there to be distortion in other hardware, before it even got into the PC.

sometimes things are quiet because people keep their distance from an insensitive microphone.
sometimes things distort because they smush their face into an already-very sensitive microphone.
sometimes things are weird because someone else used the hardware and changed something. etc.

So when doing serious recording, try to start with a sanity check: make a quick recording and listen to it (headphones are often better than speakers), and maybe check the waveform for flat, clipped tops.

(If you want to start thinking like a sound engineer there are more details to this - in particular, think about distance from microphone to source before you adjust gain levels. And then there's a long list of perfectionism you can care less about.)

More on this, and why, below.



💤 "Why do recording levels matter?"

Mainly because whenever your recording levels were overly low, the noise is implicitly somewhat higher.

If this comes down to a limitation of hardware or situation, so be it.

If this is your attention and turning one knob during recording time, this is worth that attention.


And Praat's helpfulness hides this difference.

When Praat visualizes audio, as a waveform or as a spectrogram, it will scale up whatever you already recorded.

This is great in that once you have settled your recording, then you will always see a waveform and spectrogram with whatever is the strongest signal, regardless of exactly how strong it is.

This is terrible in that these displays become a bad way to judge how strong or noisy a recording is, and whether you might want to re-record it.


Same sound, different input gain
💤 "But why do low levels make it noisy?"

In a perfect world, it wouldn't. But even the fanciest of recorders have an imperfection we call a 'noise floor': noise that is always there, and at a fixed level, no matter what you do. Very quiet, but still there.

Noise is inseparable; the moment you pressed recording you accepted that blend of signal and noise. So while you can amplify what you have afterwards, that noise comes up right along.

But since this floor happens to be fixed, you can make it nearly-irrelevant: Your job is to ensure that microphone's level is well above that noise floor (but not so high that the signal distorts).

If the microphone's levels can be adjusted to be, say, 80dB above the recording's noise floor, you are very happy.
If the adjustment was so quiet that the mic is just 20dB above the noise floor, then you are less happy.

(and yes, there are also sources of noise that will come up when you do this, such as the amplifier's, and the mic's. There's nothing you can do about that.)

The image on the right has two recordings of the same sound (two identical mics, same recording interface), but one had 45dB less input gain.

What is clearly clearly signal in one, starts falling into the background noise in the other. We can still use this particular recording, even, but it is recognizably noisier.


...but as mentioned, Praat's "let's do the best with what you have" stance

ends up putting wildly different scaling on all these waveform and spectrogram views.



💤 Another angle to illustrate the "praat shows you something regardless of how loud it actually is" effect, to yourself
two taps
  • record some gentle taps on the microphone but otherwise be quiet.
  • View & Edit the sound
you should see only the taps, and an otherwise quiet waveform
  • remove the taps from the sound: select with mouse, SoundSet selection to zero
it will suddenly show loud everywhere that isn't the zeroed-out reasons
Sound view which shows whatever is the full range in the current view
Sound view which shows whatever is the full range in the current view
  • undo that, then go to SoundSound Scaling, and in particular compare 'by whole' (the default) and 'fixed range'
  • 'by whole' - looks for the loudest sample in the whole sound
this is the detault
  • 'by window' - looks for the loudest part within the currentlyzoomed area (if stereo: max of both channels)
  • 'by window and channel' - by maximum used range (if stereo: individually)
  • 'fixed height' - seems to be "amount around calculated average"
"so if you say 2 you basically see the whole thing (but might hide a DC offset - not that you generally care about that)
  • 'fixed range' - seems to be "give min and max (average implied)"
"so if you say -1 and 1, you see the whole thing

If you set SoundSound Scaling to "by window", and then scroll around, you will get an effect like the image on the right: no matter how quiet, it will show it in full view. If you move around, it will change the scale as you scroll. The spectrogram does something similar, though is configured separately.


Viewing your recording

Sound object, View and Edit window

Select the Sound, press the View & Edit button.

More basic things you may want to do is zooming in and out, scrolling around, cutting pieces off the edge - mainly see the Edit and Time menus (and maybe learn the keyboard shortcuts)

The green, blue, and red lines and dots on the spectrogram are basic analysis of speech intensity, pitch, and formants.


💤 The spectrogram is already tuned to show voices, and this can be tweaked a little further

Spectrogram → Spectrogram settings

View Range - Normally 0Hz to 5kHz (even when you recorded more) - there's almost nothing interesting to show above that, and zooming down to this range means we can see the pitch movement is more visible. This is a good default, and you arguably could make this even narrower to see pitch movenebt even more.

Window length is about the (STFT) tradeoff between frequency resolution and time resolution. The default (0.005) is a good tradeoff for many tasks. Higher (0.015) may sometimes make e.g. separate formants more visible -- yet makes them harder to place in time precisely.

The concept of dynamic range relates the loudest to the softest levels. Praat determines the maximum in a recording, this setting determines how much lower is considered quiet enough and not worth showing. The default 70dB shows almost everything (including background noise), lowering to 50 will remove quieter noise and signal, 30 does so more aggressively. It will look cleaner but hide subtler detail.


Spectrogram → Spectrogram advanced settings

Maximum is the energy level (you can ignore the units) to treat as the loudest to show (black). By default this field is ignored, because autoscaling handles this (...within a zoom level, so scrolling will make it vary - if you want to inspect in detail, you might care to turn off autoscaling).

Pre-emphasis considers that the loudness of speech's components (vowels mostly) falls approximately -6 dB per increasing octave, so we amplify higher frequencies just for the visualisation, so that it shows them roughly equally. The default is +6dB per octave. Higher than that puts more focus on higher sounds. (identity-gain point at 1000 Hz?)

Dynamic compression you can think of as someone turning up a volume knob at points of the recording being quieter.

this has use if there is significant variation in how loud individual responses are.
...but if you have strong recordings, all this really does is take the near-silent parts between words and turns up what's there -- i.e. the background noise, which is the least interesting part.
The setting is a fraction, how much to amplify any part towards the level of everything else. You rarely want to make this higher than 0.5 or so (because that's often around 20dB).


https://www.fon.hum.uva.nl/praat/manual/Advanced_spectrogram_settings___.html

Annotating your recording

Create a TextGrid

To create an empty TextGrid of the same length as a given Sound:

  • Select Sound
  • Find the Button: AnnotateTo TextGrid

Usually you want to start with just one interval tier, in which case make it e.g. (note: useful, clear naming is great later):

All tier names:                  MyName
Which of these are point tiers:  


💤 If you want to actually understand what that form's default values...
All tier names:                  Mary John bell
Which of these are point tiers:  bell

...actually mean, read on:


You can have multiple, independent 'tracks' of information, called tiers, which is useful when there distinct things worth noting, e.g. a beep for each response, one speaker, another speaker, a speaker's broken down into phonetic detail, etc.

Note also that often want to mark things that took some time, and sometimes we mark things that are a point in time, which is why each tier can be either an

  • interval tier
consists of segments that always covers the whole recording (but many are unlabeled and thereby considered empty)
inserting something at a time will split the segment that is currently there into two
you can optionally label each segment (you might e.g. start by marking the silences)
you can select the segments
  • point tier (sometimes 'text tier')
inserting a point adds a specific points in time
you can optionally label each point
you can select the labels


So now that we grasp that, let's say we know there are two people speaking in our recording, and a beep at the start of each response.

Now look at that form again.

  • Tier names - space-separated list. Settles the amount of tiers and their tiers at the same time
  • Which of these are point tiers? - repeat the names of tiers you want to be point tiers. Any not mentioned will become interval tiers

And we can figure that:

All tier names:                  Mary John bell
Which of these are point tiers:  bell

means:

create Mary as an interval tier
create John as an interval tier
create bell as a point tier

Editing TextGrid

This article/section is a stub — some half-sorted notes, not necessarily checked, not necessarily correct. Feel free to ignore, or tell me about it.

If you did the above, you have a Sound and a TextGrid of the same time length.


Viewing and editing:

  • zooming (CtrlI and CtrlO)
  • scrolling (PgUp and PgDn) (...if not zoomed out to be looking at everything)


Editing those annotations

Once it it your task to do more than a little annotation work, it helps to get comfortable with Praat key shortcuts.


A mix of keyboard and mouse seems to be most convenient in that you can click in waveform or spectrogram:

  • mouse-click chooses a point in time,
  • mouse-drag lets you move an interval or point

...you can do a lot with keyboard only.

See the menu, but some if it includes:

  • The selection is a not-yet recorded interval which can be useful
    • Up and Down to move the whole thing left or right
    • ShiftUp and ShiftDown to move its start
    • CtrlUp anbd CtrlDown to move its start
    • ...or, in a point tier, you just move a single point, which you can make a real marker with Enter
  • AltUp and AltDown base selection on previous or next interval/point in tier
  • and then
    • Enter to split in current tier
      • in an interval tier, we split the current interval at selection start and end (whichever didn't exist already, and can't split two at once)
      • in a point tier, makes point at current position
    • Ctrl1, Ctrl2 - like Enter, but in specific/other tiers - first tier, second tier, etc...


  • Altbackspace - remove point / merge interval with previous

Other views/editors

Pitch editor

https://www.fon.hum.uva.nl/praat/manual/PitchEditor.html

How do I...

How do I shorten the length of a textgrid?

Initial Praat setup

Installing Praat

TODO

"The phonetic font is not available" (Optional, maybe)

Praat wants a font that covers all IPA characters in Unicode


Praat comes from a time where you likely needed to install SIL Doulos and/or SIL Charis to guarantee the IPA characters would show up.

These days there are other fonts that cover them, but it's still not a guarantee, and Praat still plays safe and still complains if you don't have those fonts installed.

If it works without doing that, you can ignore the warning.

If it doesn't work, or you just want it to be quiet, you could install those fonts.


See also: how to install fonts (in general)

Consider tweaking Windows (optional)

Showing file extensions

Say you have made

Recording1.wav
Recording1.Manipulation
Recording1.PitchTier

then it is fairly clear what belongs together.

However, Windows typically hides extensions (...that it knows about). This is nice for a uncluttered overview where you probably have a nice icon anyway -- but less precise when using a computer as a tool. In particular, windows explorer may just show you:

Recording1
Recording1
Recording1

which is less than great for our case.


If you want to see extensions:

Win7: Tools : Folder options : View tab : uncheck "Hide extensions for known file types"
Win10: Explorer window : View : check "File Name Extensions"
Win11: Explorer window : View : Show : "File Name Extensions"
OSX:
Linux, GNOME: shown by default

Open with

What

When you double-click a textgrid, would you want windows would open it in Praat?


Why?

Say you often find yourself opening one file at a time with Praat's OpenRead from File for each textgrid you want to open.


There are faster options (beyond writing a script to open files).

  • Write a script to open multiple files (is specific and is work, though)
  • Making your system open files with Praat by default
    • Window's "Open with" associates a file extension with a program, after which it will run Praat.exe with that file -- which each ends up working as an "add to list".
    • There are similar tricks in Linux (TODO: mention)


How?

  • Windows: Right-click a file (with an extension you want to associate with a program - you would have to do this once for each praat file extension)
"Open With..."
may require some extra clicking (varies with windows version) like "More apps", "look for more apps", scrolling, etc
The first time you do this you also need to browse to where precisely the application is located on disk. If you've done that once it should be in the list of apps

Tweaking and extending praat

For reference: The Praat preferences folder

Praat's interface is just the default.

It can be altered with configurations specific to your own account.

This is created the first time you run Praat, you can tweak this later, and Praat plugins will install themself here.

This is all in a directory placed at:

  • In Windows: %USERPROFILE%\Praat
  • In OSX: ~/Library/Preferences/Praat Prefs/
  • In Linux: ~/.praat-dir/


This contains:

  • Preferences file[11]
mostly contains a whole bunch of defaults
Most Praat menu items and buttons are can be removed, and you can add more.
You can do that on the fly, but if you want that permanent for your setup, this is where to add that.
  • possible plugins, in directories -- see below


https://www.fon.hum.uva.nl/praat/manual/preferences_folder.html



More practically, how do I put more functionality in my installation of Praat?

There are roughly two options:


Run commands that happen to alter your own buttons file permanently

If you run Add action command: and Add menu command: yourself or from a regular script, it will permanently alter your personal buttons file
You can also put that in a script, which will usually be called something like "run once to install this"

Upsides:

  • nice way to incrementally alter praat Praat to do all the things you want, the way you want it

Downsides:

  • over enough time you won't remember all the things you did to get here


Put plugins in your Praat profile (Plugins)

Alternatively, place that exact same file in a new directory under your Praat preferences folder.

This will now be run at Praat startup, but not alter your buttons file.

Upsides:

  • each of these is more self-contained, and more easily copied between installs
  • picking up what's there at each run is easier for development

Downsides:

  • more up-front work



Site installs?

Installing things for everyone who uses praat on a computer is not a thing that Praat itself does.

The closest options are probably

  • have every user install Praat from a packaged install (that also installs these extras in your profile)
  • have users install Praat from a site-managed install (same idea, but managed by your IT department)
  • have people visit a shared folder and do a one-time install (e.g. a Praat script that registers a menu item pointing to that shared folder)



Praat plugin notes

Praat plugins, at a lower level, amounts to "if there is a specific file in a specific directory in your preferences, Praat will execute it every time it starts up"

For example, if I

  • create a directory called plugin_annoying
  • create a file in it called Setup.praat containing:
beginPause: "This is an annoying window"
clicked = endPause: "Okay", "Well done", 2

...then I will get an annoying startup window every time I start Praat.


Usually what plugins do is add some menu items, pointing at specific useful praat scripts.


https://www.fon.hum.uva.nl/praat/manual/plug-ins.html




Plugin and toolkit examples

Praat 'toolkits' aren't really special, but we tend to call plugins that if they are a more interesting, coherent collection of tools.

...which may not always be installed as plugins, just "does a useful thing when you run this .praat"


Praat vocal toolkit

Praat Vocal Toolkit [13]



ProsodyPro notes

http://www.homepages.ucl.ac.uk/~uclyyix/ProsodyPro/

Not a plugin in the sense of 'adds buttons to the interface', more of a script that when run, initiates a semi-automated annotation.

Unsorted

Praat scripting notes

Data types and structures

Basic types, an structures involving them

String operations

More on vectors

String vectors

More on matrices

Types and GUI elements

On paths

Control, and reuse

For loops

conditions

Procedures

Calling commands

Forms

Differences between using procedures, forms, objects, and scripts

Objects and selections

Logging to the info window

We generally use appendInfo, appendInfoLine - add to window content, without or with going to the next line

useful if you want to build up a single line of output from distinct parts of your code


People may well clear the info window at the start of a script, often using clearinfo

(if scripts write out useful things to check later, clearing is not always ideal if you script may ever be called by other scripts, but you/others can worry about that at that time. For quick scripts it can keep the output clean, though.).


There is also writeInfo, writeInfoLine which 'replaces all info window content with this (again, without or with going to the next line)

(again, overwriting might be more annoying than useful for debug reasons)


You can hand in multiple things to be printed.

For example:

appendInfoLine: minimum, tab$, maximum

is equivalent to:

 appendInfo: minimum, tab$, maximum, newline$


See also:

Running other things

Running Praat scripts from Praat

Running external commands from Praat

Other things telling Praat to do things

Calling into Praat executable should typically be done with --open, --run, or --send


Opening files with Praat can be done like

Praat --open data\hello.wav data\hello.TextGrid
Praat --open script.Praat



You can

Praat --run testCommandLineCalls.praat "argument"

...which does so without a GUI; any Info-window output goes to the console that runs it.


To command a running Praat GUI (or a new one if one wasn't running), you want sending

Praat --send "command"

sendpraat is roughly the same, e.g.

sendpraat 1000 praat "Read from file... hello.wav" "Play reverse" "Remove"


https://www.fon.hum.uva.nl/praat/manual/Scripting_6_9__Calling_from_the_command_line.html


Things that work on the same files, but do not interact

praatio

parselmouth

Files

Text files, short text files, binary files

Many data-style objects (including some cases you may never use, like Sound objects) have a structured representation that can be saved as

  • a text file
which contains a little more than necessary but is human-readable.
  • a short text file, which basically omits the variable names,
but is stable enough that parsers should have no trouble (no idea if there were breaking changes over time)
  • a binary format, which is a little more compact.


Some thing have futher forms, e.g.

  • PitchTier and DurationTier has
    • PitchTier/DurationTier Spreadsheet file (not unlike their short text form)
    • headerless spreadsheet file (basically TSV)


While the text formats look parseable by yourself, try to avoid that when it is easy because the format has changed. Praat will know how to handle that, but your or other's libraries may break over time.


Extending praat

_

Drawing

See also

https://www.fon.hum.uva.nl/praat/manual/Scripting.html


Unsorted