Saturday, 6 October 2018

MEASUREMENTS: RME ADI-2 Pro FS as DAC (Part II - frequency response, noise, distortion and jitter)


As we saw previously, the RME ADI-2 Pro FS is based on the AKM AK4490 DAC chip and provides a good assortment of features like filter settings. We've seen that the headphone output amplifier provides plenty of power and the beginnings of what looks like a highly accurate DAC; consistent with my subjective opinion.

Time for part 2 of the objective evaluation today as we continue to explore the DAC output quality.

0. Preamble

Before continuing, let's just lay out things like firmware versions and how the device was measured.

For RightMark testing, I'm using the current version of RightMark PRO 6.4.5 with the multitone frequency response set at 20Hz to 20kHz. For measuring the ADI-2 Pro FS DAC itself, I'll do a "loopback" setup where the DAC analogue outs are plugged directly back to the ADC inputs. Measurement results of other devices for comparison will be the usual playback from a source like the Raspberry Pi 3 B+ "Touch" to DAC and measured with the RME's ADC.
ADI-2 Pro FS LOOPBACK: Win 10 laptop --> 12' USB cable --> RME ADI-2 Pro FS --> 6' XLR cable --> RME ADI-2 Pro FS XLR input --> same 12' USB cable --> same Win 10 laptop
OTHER DACs: Pi 3 B+ "Touch" --> 6' USB --> comparison DAC (eg. Oppo UDP-205, TEAC UD-501) --> 6' XLR / RCA cable to RCA-TS converter --> RME ADI-2 Pro FS --> 12' USB cable --> Win 10 measurement laptop
PonoPlayer: Test signals on PonoPlayer --> 6' phono-to-RCA --> RCA-to-XLR --> RME ADI-2 Pro FS --> 12' USB cable --> Win 10 measurement laptop
For S/PDIF input to ADI-2 Pro FS:
Squeezebox Touch --> 6' TosLink / Coaxial generic cable --> RME ADI-2 Pro FS S/PDIF in --> 6' XLR cable output --> RME ADI-2 Pro FS in --> 12' USB cable --> Win 10 measurement laptop
The RME ADI-2 Pro FS was updated to the 180/88 firmware (there has since been version 183/88 which improves USB compatibility but would not have changed the sound quality measured here). I've set the ADC to "Sharp" filter mode for these tests. As for DAC filter, unless specified, the filter is set to the default "SD Sharp" (I will have a few comparisons with "Sharp" and "NOS" filter settings as well).

I. RightMark Testing

A. 16/44

Since the set-up is in loopback mode for the RME ADI-2 Pro FS, I have to keep the samplerate and bit-depth the same, the ADI-2 Pro loopback measurements are therefore done totally in 16/44. For the other devices, even though the test signal is always at 16/44, the ADC was set to 24/44. In any event, we see that 16/44 is no problem especially for the DACs with XLR output resulting in low distortion level throughout. For fun, I've included the old Squeezebox Touch and PonoPlayer measurements in the mix :-). Neither of these are operating in balanced mode and you can see that the distortion results are a little bit higher.

Charts:

Really not much to differentiate the devices with a 16-bit signal. However, it looks like the PonoPlayer has more crosstalk and higher IMD+N even compared to the Squeezebox Touch (for crosstalk, this could just be a cable/adaptor difference).

You see that I'm also showing the differences in frequency response between the ADI-2 Pro FS's "NOS" setting compared to the default "SD Sharp" and "Sharp" filters. Also, the PonoPlayer using the Ayre "Listen" filter dips off early before 20kHz similar to the NOS filter setting. The only other non-flat curve is the Squeezebox Touch - no big deal, just a 0.7dB dip into 20kHz although there seems to be a bit more ripple in the high frequency response.

B. 24/96


Going into "high resolution audio", we see representatives from the 3 "families" of major DAC chip manufacturers in a little showdown. Of course, DAC quality is not just about the chips themselves but also analogue output stage, power supply, etc..., the converter obviously is the heart of the process and has a significant role to play.

To the left is the ADI-2 Pro FS with AKM's AK4490 "Verita" inside. Then we have the Oppo UDP-205 with its ESS ES9038Pro; on paper at least, likely the highest spec'ed consumer DAC chip these days. Then after that we have the TEAC UD-501 running dual TI/Burr Brown PCM1795; this chip is almost a decade old at this point, released in 2009. Notice how each device achieves excellent low noise floor. Really the only differentiating factor is the amount of measured distortion! In this regard, if we look at the XLR measurements for best results, the Oppo / ESS ES9038Pro does measure better than the RME / AKM AK4490 or TEAC / TI PCM1795 DACs by a miniscule amount - more than likely representing the lower limit "floor" of the ADC. Seriously though, high resolution DAC technology has achieved a level of accuracy that is well beyond the acuity of mortal ears at this point in history, so other than "golden ear" claims, I believe the vast majority of people would describe the sound as much more alike than different, allowing for subjective factors like mood and non-utilitarian factors in the assessment process.

A beautiful 20-bits resolution measured for the RME ADI-2 Pro FS balanced and unbalanced outputs.

We see some variability in the unbalanced measurements. Remember that for crosstalk measurements, much of the variability is likely due to the cable/adaptor used. On the whole, if you want the lowest noise and distortion, make sure to use some good balanced cables (Mogami Gold and inexpensive Monoprice Premier in my main system these days).


Looks like both the Oppo and TEAC picked up a bit of 60Hz hum when measuring the unbalanced RCA output which the RME did not using the single-ended TS out (remember, it's also protected in loopback mode).

Notice that at the 96kHz sample rate, even though the PonoPlayer still has an earlier high frequency roll-off compared to the other devices, at 20kHz, it's only around -0.1dB. Remember this if you try listening with the Pono "Revealer"; my belief is that if you hear a difference between hi-res and 44.1kHz, it's more likely than not just the difference in high frequency roll-off... An effect that they created because of the type of digital filter they chose!

C. 24/192


24/192 looks pretty much like the results we get with 24/96. Again, the RME, Oppo, and TEAC are putting up great results with the Oppo / ESS ES9038Pro remaining as the "king" of low. Again, there's some 60Hz hum when we look at the unbalanced connection from the Oppo and TEAC.


D. 24/384
But wait! We're not done :-). Since the ADI-2 Pro FS can go higher as a DAC, RightMark allows us to test at the 24/352.8 & 24/384 settings:


Looks great. Not sure how many of you out there use these very high sample rates! As expected, no difference whether one uses a standard "SD Sharp" filter or "NOS" at these kinds of sample rates.

I'm not sure exactly how the AKM DAC chips work, but I know that with the TEAC (TI/BB PCM1795), at 384kHz, the digital filter is turned off. [Addendum: Looks like AKM switches to a fixed "Slow" filter above 192kHz. See the Addendum a paragraph down.]



We see a little bit of noise from the DAC at ~100kHz (low level, below -105dBFS). Otherwise, the noise floor is remarkably clean to almost 200kHz. Remember folks, this is with a standard stock switching power supply to the DAC/ADC and the measurement computer plugged into its switching AC adaptor capturing this information via USB. I still see all kinds of beliefs expressed in audiophile forums suggesting that linear power supplies are somehow less noisy or improve sound quality. So where's the evidence for such a belief when used with low-power devices like DACs and digital streamers?

[Addendum: If you look at page 85 of the ADI-2 Pro FS manual, there is a frequency response curve being implemented by RME to extend the output for an even flatter curve at 384kHz - essentially flat out to 100kHz (-0.25dB) to compensate for that "Slow" AKM filter noted above! I'm not seeing that in my sample here. I've been told that this is being worked on and I'll update this when a future firmware addresses this. Note that this is of course not in any way a concern for the listener, rather more for those who want to use this device for measurements and want/need the ability to produce flat ultrasonic frequencies even beyond the limits of the 192kHz samplerate curve.]

E. Comparison of Asynchronous USB2.0 and S/PDIF TosLink & Coaxial Inputs
Remember the ADI-2 Pro FS has a TosLink input on the rear panel, and with some breakout cables, coaxial S/PDIF and AES/EBU input as well.

Using my trusty old Squeezebox Touch with its TosLink and Coaxial outputs, I connected them to the RME for these 24/96 results from the XLR output:


Absolutely underwhelming, right? :-) These measurements are identical.

Whether asynchronous USB, digital S/PDIF TosLink or coaxial, this is bit-perfect folks... For a high quality DAC these days, there should be essentially no difference just like what we see here. The USB input is being fed by a Core i5 laptop computer with the test signal generated in the machine, while the TosLink and Coaxial inputs are being fed by an old Squeezebox Touch connected by WiFi to my server computer in another room of the house run off an i7 Windows Server 2016 machine with Logitech Media Server running in a Linux virtual machine. Sure, maybe later on we can see some jitter difference between TosLink/Coax/USB, but even then, in modern devices, the jitter will be minimal and IMO insignificant (as discussed/demo'ed recently).

Now for the just-as-underwhelming graphs...


Seriously folks, if I may go on a related tangent for a moment... It's absolutely ridiculous and uninsightful for audiophile writers like this to claim that they hear some kind of big difference between digital interfaces suggesting that somehow USB input has "a bit of an underlying softness, grayness, or lack of dynamics" as if one can generally claim such a phenomenon. Is it that he was so unlucky to have tried many poor quality USB DACs or rather this claim is all simply subjective expectation bias against USB!? All the more disappointing considering that this was published on InnerFidelity where until not long ago, it was certainly the most balanced and reasonable (as well as IMO most respectable) of the Stereophile-family of websites. Oh well...

By the way, even though I've not seen a role for consumers, the ADI-2 Pro also accepts 2-channel ADAT signals over the optical interface up to 192kHz.

II. SpectraPLUS - THD(+N) & THD+N vs. Frequency

With RightMark above, we've seen the IMD+N sweeps. But as described a few weeks back, using SpectraPLUS, we can run some traditional 0dBFS 1kHz THD(+N) measurements at 24kHz bandwidth as well as my "THD+N Matrix" at different amplitude levels to show changes in the noise floor and harmonics up to 48kHz bandwidth:

"THD+N Matrix"
Nice looking 0dBFS THD(+N) graph (recorded at -3dBFS on the ADC) with very low 0.00016% or around -116dB distortion using my "standard procedure". Equivalent results across the two stereo channels. The harmonics have a typical structure to them with gradually declining higher order levels.

Notice that I blurred out the THD+N (and by extension SINAD) results with the lower level 1kHz signals because of what appears to be inaccurate calculation of the noise level. I'm awaiting a response from SpectraPLUS to double check about this.

The only anomalies on the "THD+N Matrix" were the low-level bursts of noise/"needling" in the -15 and -30dBFS FFTs at higher frequencies. I have been told that these are not uncommon with delta-sigma DACs in general.

Using SpectraPLUS in loopback mode, we can also do a THD+N sweep across 20Hz to 20kHz ("SD Sharp" filter):


Nice and low levels throughout. I also did the same THD+N sweep using the unbalanced TS output and the graph looked almost exactly the same so I didn't bother reproducing it here.

III. Jitter

Time to talk about our favourite topic :-). I'll be using the high-resolution SpectraPLUS software with 1M-point FFT to show these results. For the S/PDIF inputs, I did not engage the sample rate conversion (SRC) ability of the ADI-2 Pro FS.


Here's the 24-bit J-Test:


And let's accelerate that 24/48 test to 24/96...


Boring, right? :-)

Actually, this is quite remarkable because there is no difference in jitter performance whether one is using the asynchronous USB input or either of the S/PDIF interfaces (I don't have any device hooked up to try the AES/EBU digital in)! Excellent performance from RME, even better than the Oppo UDP-205 in fact; have a look at that Oppo measurement a couple weeks ago and notice the various non-jitter sidebands around the fundamental frequency. It's particularly gratifying to see that the TosLink optical interface remained jitter-free since on many devices it tends to be worse than coaxial and USB.

The upgrade to "SteadyClock FS" was the main difference from the previous "non-FS" model so if I were to compare the devices, it would be these jitter graphs that would show significant improvements. (See the manual on page 75 showing the improvement in "skirting" between the FS and non-FS models. Honestly, that small difference below -100dBFS is not going to be audible with music playing!)

Remember again that the DAC is not only being fed a signal from different digital interfaces (USB vs. TosLink vs. Coax), but also different machines. There's a world of difference between a modern Intel Core i5 laptop with 8GB RAM and SSD storage and the Squeezebox/Logitech Touch based on an old ARM processor streaming the signal over WiFi! Yet whether it's the identical results in the RightMark test (Section I.E.) above, or essentially identical results here with the jitter signal, the digital interface is clearly robust and free from temporal inaccuracies even between disparate machines, interfaces, and cables with this converter.

IV. Conclusions

Well folks, there you have it. Measurements of the ADC side of the RME ADI-2 Pro FS a month back, a discussion of output levels and filters a couple weeks back, and today, finishing off with other aspects of the DAC performance.

This is clearly an advanced professional package with excellent objective and subjective performance in a small box while still featuring a nice IPS LCD display and sophisticated collection of buttons and knobs for entering settings. Realize I've only been discussing measurements as they pertain to basic ADC/DAC functions. The ADC noise level is superb even out to 768kHz. The implementation of discrete reference levels is a big plus here to optimize dynamic range across wider input levels for my measurement purposes. As for the DAC, we have an assortment of filter settings including "NOS" for those so inclined. The measured 20-bit dynamic range is superb. There's a good +2dBFS overhead for intersample peaks discussed last week. Distortions are extremely low across samplerates (measured up to 384kHz). And there's essentially no jitter to speak of regardless of USB or S/PDIF digital input source!

So far, I have not even touched on the DSP features built into the box such as the 5-band parametric equalizer, bass/treble levels, a "loudness" feature to compensate for frequency sensitivities across volume levels, and the headphone crossfeed function. Nor have I touched on the digital audio stream metering and analysis tool (DIGICheck) provided by RME. Indeed, the breadth and depth of the features incorporated into this device is more than likely beyond the needs of most home audiophiles... But you never know when some of these features might just come in handy! :-) Consider too that we're looking at a device that costs around US$2000 for both a high quality ADC and DAC. In the audiophile world, that's certainly huge value for the money!

I would like to measure the quality of the headphone outputs at some point when I have time and maybe run some DSD tests as well if I can. It might be fun at some point to get balanced cables for my Sennheiser HD800 and take advantage of this feature of the ADI-2 Pro FS also. Given what I have heard and the results I have seen, I'm of course expecting excellent quality.

As far as I can tell both in my listening sessions over the speakers and headphones plus from the objective measurements, this device literally features "textbook" transparency. From my perspective as a "more objective" audiophile, the achievement of transparency is the highest accolade I can give to any high-fidelity device.

BTW, remember that my interest in the ADI-2 Pro is actually mainly so I can have a high quality ADC to be used in future measurements; in this regard, I'm of course very impressed already! If I were mainly interested in the DAC playback, I would have a good look at the RME ADI-2 DAC FS; a "DAC-only" consumer-oriented package with IEM headphone output featuring even lower noise level, remote controller, RCA output, and all black aesthetics.

------------------------------------------

Album of the week while working on this blog post - Janelle Monáe's Dirty Computer. It also comes with a Youtube futuristic dystopian "context" video. For those interested in modern pop of course :-)



Kudos to Reprise Records for getting the recent Tom Petty An American Treasure (Deluxe) done right! Check out the DR Database and compare the 16-bit (average DR8) and 24-bit (average DR11) versions.

Now that is what I was hoping for would happen eventually when I wrote about "The Wisdom of Simplicity" back in late 2015. Use a higher dynamic range version for the hi-res 24-bit and vinyl mastering while the more compressed version can be used for CD/iTune/streaming. This will sonically differentiate the higher quality master. The hi-res version can then be appropriately promoted as having the superior sound, and listeners can be "taught" to recognize the difference and be more discerning as consumers... In time championing an elevated standard of sound quality rather than perpetuate the "loudness war".

Here's a comparison of the 16-bit vs. 24-bit versions of "Into The Great Wide Open" (Live at Oakland-Alameda Coliseum) from the album:


Until next time, enjoy the music everyone! And a happy Canadian Thanksgiving :-).

20 comments:

  1. congrats, i'm owner of adi-2 too
    now i try to find some fair jitter measurment tool with digit value not only spectrum view

    ReplyDelete
    Replies
    1. If you find one please post it here. I've never found one myself.

      Delete
    2. Hi guys,
      Yeah, I haven't seen a test that spat out a number since the days when Stereophile used the Miller Audio test. The problem with a number is that it doesn't necessarily capture audibility. Maybe some composite number measuring the jitter sideband peaks and scaling them to distance away from the fundamental might help?

      Delete
  2. Hi, thanks for your latest blog post from one avid reader.

    I just want to comment on the last 2 paragraphs. I see that you apparently have faith in the TTDR tool. I was hoping that my previous warning(s) and debunking of that tool had put you off it. Or at least given you pause before posting online to share what you learned from it about versions of an album.

    The problem is that it gives strongly different DR numbers for versions that have no difference in dynamics or compression, but have had some irrelevant attribute changed in the mix or master. The classic and ubiquitous example being bass channel separation and phase. Perhaps email me if interested in discussion. But I have already covered it in previous posts where you refer to it. Slightly disappointed in myself for failing to get the message across well enough.

    cheers

    ReplyDelete
    Replies
    1. Hi tnargs,
      Thanks for the note and I agree that we can't put too much faith in the TTDR values only. Note that the DR Database results were not submitted by myself.

      What I can say though is that the 24-bit version is clearly different and sounds IMO better; just turn up the volume on a good sound system! Especially for live material which I've always preferred a more dynamic sound.

      I've now included in the post a comparison of the 16-bit and 24-bit versions showing what the DR meter was measuring (both channels overlaid hence the green and bluish colors) - for this track TTDR is measuring DR8 for the 16-bit version, and DR11 for 24-bit. That's the same as the album as a whole.

      Delete
    2. The problem is aggravated by the lack of understanding of the nature of jitter and its effect on auditory. For example, in the picture of harmonic distortion, not only absolute numbers are important, but also the order of harmonics. Does it make sense to measure jitter in a digital domain, when clocks are specified in different modes (in one case, this is the receiver, in the other - the transmitter).
      I think we are all at the beginning of a long way of developing the procedure for measuring and correlating the results with listening experience. For example, the Sharp mode gives the best response, but I still prefer to listen to music in the NOS mode. At the same time, I cannot be called a fan of softened sound, (and, by the way, Karsten specially developed an equalizer preset for curve correction on the NOS mode). Perhaps we are talking about some temporal characteristics of the signal. Without a filters, maybe it unfolds in a more advantageous way for the my ear. But we try to judge this in the framework of electrical engineering measurement standards of the 50s?
      The RMAA measurements provide an indication of the lack of technical problems in the device, rather than the ability of the component to play music well. These are not my words, this is a quote from one of the creators of this program, with whom I am in correspondence. Same thing as high DR is not a warranty of well recorded music.

      Delete
    3. Hi Adova,
      Sure, it's possible that things like the technical measurements don't correlate with perceptual preference. But I'm not sure I've personally experienced this dissociation much. I have not generally felt that NOS sounds better than a sharp filter for example.

      I have no problem with a general perspective that there are tests that we can run to evaluate sound quality which could better correlate with perceptual preference. That's why I prefer posting things like the full J-Test spectrum rather than a single number (assuming there is easily accessible software) and why I think it's interesting to show the harmonic distortion "matrix" with signals below 0dBFS.

      My suspicion is that what is "missing" are not measurements of the devices! It's more a question of individual preferences - human idiosyncrasies. Some prefer a more technically "clean" device, others find joy in even order harmonics... That's fine.

      Delete
    4. The sinusoid is not a sound signal, and therefore does not reflect the behavior of the system under study when playing music. For essentially non-linear systems, the superposition principle is not fulfilled. That is, the response of the system to a complex signal is not equal to the sum of reactions to the components of this signal. That is one harmonic alone. Two sines - already 10 times more modulation products. 100 sines 1000 times more modulation products. The condition is not satisfied that the response to 1 sine will be the same as the musical signal.
      So it is not just a lack of information, but also a lack of investment in research into the audio industry compared to military or medical technology.

      Delete
  3. Thanks for the apt comments on the improved DR of the 24/96 American Treasure Deluxe. Excellent. But I've still gotta gripe that this many years into the hi-res era, you don't get a scrap of box set art or liner notes for your audiophile premium dollars — no PDF, no nothing.

    ReplyDelete
    Replies
    1. Yeah Hal,
      Agreed. Sucks. At around $50 on HDtracks, should at least have a few PDF pages!

      "Mr./Ms./Mrs. Recordperson, can you spare a PDF???"

      Delete
    2. Jesus, i don't know how many times i contacted the labels asking for the missing PDF booklet. Warner music: never answered. Decca and Deutsche Grammophon (or Universal music, if you prefer): never answered. +90% of the music i buy is classical, so booklets with details about the music, composer, artists, conductor, orchestra and technical recording details are fundamental to me!

      My "modus operandi" about buying downloads these days is to buy or favor always the label that provides the PDF booklet. And when i cannot find anything, i look in foruns if someone didn't have a scan of the booklet to share (yes, that's right, to SHARE) and then i buy the download.

      Delete
  4. Hi Archimago.

    As always nice article. We can conclude more and more that the digital part (and even analog part) of our systems aren't the weak point (just remembered that Innerfidelity recently made an interview with Jason Stoddard, and he makes this point very clear, stating that the major problem in an audio system is the speakers, not amplifiers, and much less dac's). BTW, my favorite part of that interview was when Brian Hunter asks Jason Stoddard if it is possible to make a good sounding amplifier without hearing, just by measuring it, and Jason cut right to the chase saying: "Yes". Remarkable :D

    Changing the subject, the only thing i think is wrong is to differentiate a 44.1/16 from a 96/24 recording using different dynamic range. That's misleading, since the audio difference is not about the extra frequency response and extra bits, but is being artificially generated. But anyway, it's just my opinion, i'm not here to fight ;)

    Forgive me if today i wasn't very clear in my english. Insomnia induced by bipolar disorder can be a real thorn in the flesh.

    As always best regards and keep "feeding" us with your rational audio tests! Take care!

    ReplyDelete
    Replies
    1. Ahhh, those Schiit guys :-). Cut to the chase, and one of the first to say they won't support MQA... Love it.

      True, dynamic range measurement in itself is clearly not the differentiator between 16/44 and any other form of high-res, even 24/48 for example.

      I see the dynamic range (and we're talking true dynamic range as in more dynamic mastering) as simply a proxy for music that could benefit from high-resolution.

      If an album has low values like DR8, it would be very hard to argue that going from 16-bits to 24-bits "hi-res" could yield any benefits. This is why from the beginning when I wrote the article about "High-Resolution Audio Expectations":
      http://archimago.blogspot.com/2014/03/musings-high-resolution-audio.html

      I made the point about true high-resolution music should have a decently high DR to even suggest that I would consider spending extra money on it.

      Over the years, if we look around at high-res digital music recorded/mixed/mastered well such as albums from 2L, AIX, Reference Recordings, Channel Classics, some of the Chesky, worth buying as the "real deal", invariably we're looking at >DR10.

      Delete
    2. I'm not qualified in any sense with regards to signal processing or sampling theory, so please forgive me if this question appears ignorant. I have seen people say several times that increase in bit-rate only serves to increase dynamic range, and therefore doesn't make a difference in most music which has a limited dynamic range anyway. But doesn't an increase in bit-rate reduce quantization errors? Does that not represent greater accuracy of reproduction even where the music itself is recorded at reasonably constant and high levels? Although quantization errors are referred to as 'noise', this seems quite different to say tape hiss. I fully realise that I may well be missing something important here....

      Delete
    3. Hi Unknown,
      Not much time at present to speak more of this...

      But I would suggest reading this:
      https://hometheaterhifi.com/technical/technical-reviews/audiophiles-guide-quantization-error-dithering-noise-shaping/

      There's discussion of quantization noise, dithering, etc. Very well done.

      Delete
  5. Where Focusrite Forte? you promised...

    ReplyDelete
    Replies
    1. ?? Piopi ??

      Which Focusrite Forte measurements?

      Delete
  6. Thank you for your awesome testing. I have a question about one of the results, though. In the C. 24/192 chart the teac has a dramatic drop in channel separation with the use of XLR output. Is this an error, or is this something that actually occurs with the Teac? I have a Teac ud-301 dac which appears to be a cheaper version of the 501 but has the same DAC chips, dual mono design etc, so it might be similar in performance. I'm wondering if there is anything obvious that may need attention with regard to this issue (if it isn't just a chart error). Thanks very much for all your hard work!

    ReplyDelete
    Replies
    1. Thanks for the note Jeff,
      Good catch. When I have time I'll have a look at that again. Why the drop in the TEAC UD-501 XLR at 24/192. I suspect if must have been an error as there is no reason for this to happen...

      Delete
    2. Re-did the TEAC XLR 24/192 run.

      Yup. Error fixed.

      Delete