Researchers at MIT Media Lab have found a genuinely jaw-dropping use for the LiDAR sensor sitting inside your iPhone and iPad Pro. It can detect and track objects that are completely outside the camera’s field of view. Yes, that means seeing around corners.
This kind of imaging, called non-line-of-sight (NLOS) imaging, is not a new concept. But past demonstrations relied on powerful, expensive lab-grade lasers with little application in the real world.
What makes this research exciting is that the MIT team pulled it off using the same low-power LiDAR sensor already embedded in our smartphones.
How does it work?
The team is using the LiDAR sensor to allow us to look beyond corners at objects that are not directly in our line of sight. The secret sauce is motion. As your device moves, the system simultaneously tracks the object’s shape, the object’s position, and the camera’s position over time.
Aaron Young / Siddharth Somasundaram / MIT Media Lab
The team calls this an aperture sampling model, and it essentially stitches together a series of noisy, imperfect readings into something meaningful. The outputs are not crisp photos of what is hiding around the corner. Instead, you get progressively richer inferences. The system can tell you something is there, how it is moving, and what shape it roughly has. Think of it like echolocation, but with light.
LiDAR scan vs actual / MIT Media Lab
What can it actually do?
The team demonstrated four specific capabilities: tracking a single object, reconstructing its shape, tracking multiple objects at once, and something particularly interesting for robotics, which is camera self-localization using hidden landmarks.
That last one is a big deal. A robot or autonomous system that can orient itself using objects it cannot directly see has a massive advantage in the real world. It can also help improve the self-driving tech or delivery drones for things like accident avoidance.
Sadly, you cannot try this on your smartphone right now, “as that would require these companies to release their raw data, which they often don’t do,” said Siddharth Somasundaram, one of the researchers on this project. That said, the researchers have made their code publicly available, and the sensor hardware can be assembled for under $50.
Modern displays are amazing when it comes to detail, brightness, color, and all the ingredients that make for an impressive picture—except motion clarity.
CRT screens are still the king of motion clarity, but plasma flat-panel screens hold a respectable second place, and in many ways I still miss my old 720p 51-inch plasma TV and the crisp motion I gave up by switching to a 4K LCD.
Plasma solved motion the “right” way
Plasma displays didn’t just show an image—they flashed it.
While they operate on different principles, CRTs and plasma TVs have a few things in common. First, the phosphors used by CRTs and plasma displays are the same. Second, because these phosphors fade quickly, they need to be continuously refreshed.
In a CRT, the electron beam scanning from the top to the bottom of the screen achieves this, and in a plasma, a high-speed electric pulse does the same. Because of this rapid pulse-and-fade, these screen technologies have crisp perceptual motion, since our brains tend to interpret moving images that don’t pulse as “smearing” across our retinas.
The pulsing nature of plasma technology isn’t the only reason for its better motion reproduction. These screens also have very low latency and very fast pixel response times. Combined, it’s not quite as good as CRT motion handling, but it’s significantly better than LCD and OLED technology, even today.
Modern TVs rely on sample-and-hold—and that’s the problem
Stand and deliver blurry images
Modern LCD and OLED televisions are “sample and hold” technologies. They can hold each frame of video perfectly for the entire duration of that frame without deviating in brightness and then instantly snap to the next frame without any dipping to black in-between.
On paper, this sounds like a good thing, but your eyes don’t stay still when tracking motion. As they follow a moving object, the image being held on screen effectively drags across your retina, creating the perception of blur. Even if the panel itself is perfectly sharp.
You might not even realize how blurry motion is on modern displays if all you’ve ever seen with the naked eye is an LCD or plasma. However, if you see a CRT or plasma in person, the difference is quite striking.
The sample and hold issue means that no matter how much you increase the refresh rate, that type of blur persists. It’s why my 85Hz CRT monitor is clearly less blurry in motion than my 240Hz LCD monitor. It’s especially apparent when you’re playing 2D games that scroll the entire screen, with LCDs or OLEDs smearing the image in a way that gives me a bit of a headache if I’m being honest.
Credit: Sydney Louw Butler/Shutterstock.com
It creates this weird situation where a modern TV can be incredibly sharp in a freeze frame but somehow look softer than a lower-resolution display that isn’t sample and hold as soon as you press play.
Motion interpolation is a workaround, not a solution
It’s an abomination, that’s what it is
One of the “fixes” that TV makers came up with to reduce unwanted motion blur is a technology known as frame interpolation, or more commonly “motion smoothing.” Here an algorithm creates fake frames that guess at what the middle step of motion would look like if it were captured. This creates a high frame-rate video output, which we see as smoother and more crisp.
While this doesn’t take away sample-and-hold blur, it does improve motion clarity. Unfortunately, it also destroys the intended frame rate that shows and movies were meant to be seen at. It’s also useless for video games, because it introduces an enormous amount of input lag. NVIDIA’s DLSS technology is also frame interpolation, but it works for games because of several mitigations NVIDIA put into the technology. These measures don’t exist on TVs.
While some people think motion smoothing isn’t all bad, TV makers are no longer activating it by default as much anymore, and my advice is to always turn it off because the trade-offs are just not worth it.
7/10
Brand
TCL
Display Size
85-inches
The 2025 model TCL QM6K Google TV delivers a stunningly clear and bright picture with a new Mini-LED panel, improved local dimming zones, Dolby Vision IQ, and a neat new Halo Control system for improved visuals. Get this TV and elevate your living room.
Black frame insertion tries to recreate plasma—but comes with trade-offs
Who turned out the lights?
The other trick sample-and-hold screens have to mimic what CRTs and plasma TVs do naturally is called BFI, or Black Frame Insertion. As the name suggests, the display inserts a full black frame between every original frame. This provides an instant and dramatic increase in motion clarity. However, it also has a big impact on brightness. As much as half of the light is now gone, so the image is much dimmer. Pushing overall brightness to compensate makes things hotter and more energy-hungry.
Some BFI implementations cause visible flicker, for which I personally have no tolerance at all, but the biggest problem here is that BFI doesn’t have the smooth pulsing roll off of the phosphors used in CRTs and plasma.
The future might circle back—but we’re not there yet
That might be changing, however, because a new generation of LCDs can leverage the power of multi-zone backlight technology to strobe the backlight across the screen in a way that mimics a CRT scanline.
NVIDIA’s G-SYNC Pulsar has received rave reviews from the biggest motion blur haters, and I sincerely hope that a similar technology becomes standard in TVs going ahead, so we can go back to enjoying the crisp motion we used to have without all the compromises.
To provide the best experiences, we use technologies like cookies to store and/or access device information. Consenting to these technologies will allow us to process data such as browsing behavior or unique IDs on this site. Not consenting or withdrawing consent, may adversely affect certain features and functions.
Functional
Always active
The technical storage or access is strictly necessary for the legitimate purpose of enabling the use of a specific service explicitly requested by the subscriber or user, or for the sole purpose of carrying out the transmission of a communication over an electronic communications network.
Preferences
The technical storage or access is necessary for the legitimate purpose of storing preferences that are not requested by the subscriber or user.
Statistics
The technical storage or access that is used exclusively for statistical purposes.The technical storage or access that is used exclusively for anonymous statistical purposes. Without a subpoena, voluntary compliance on the part of your Internet Service Provider, or additional records from a third party, information stored or retrieved for this purpose alone cannot usually be used to identify you.
Marketing
The technical storage or access is required to create user profiles to send advertising, or to track the user on a website or across several websites for similar marketing purposes.