Apple has proposed an Apple Pencil-like stylus that could be used with Apple Vision Pro to convey the texture of virtual objects through haptic feedback.
There’s an old children’s toy where one pencil is connected to another such that drawing with the first pencil also moves the second. A newly-granted patent is akin to this idea, except instead of copying or enlarging drawings, the Apple Pencil relays “surface roughness and other surface features, as well as motion data (e.g., trajectory and velocity) of the hand-held device” in virtual reality.
This “Surface texture detection and emulation” patent does explicitly refer to a “pencil” system, but it’s really applicable to any handheld device. Whatever form it takes, this is most probably an Apple Vision Pro accessory that Apple says is specifically intended to create “increased immersion during XR [Extended Reality] sessions.”
That means both being able to detect the texture of a virtual object, and convey that to the user. Apple describes multiple ways of doing this, such as including “a gyroscope that may be used to alter the feeling of a surface depending on the roll angle of the device.”
There could also be haptic feedback which can “create, emulate, or cancel an experience such as friction, texture, or roughness on a surface by applying forces, vibrations, or motions to the user of the device.”
Over almost 12,000 words and across nine pages of drawings, Apple also describes linear actuator components that could “extend or contract a portion of the device.” Then there could be an inertial measurement unit, and color sensors.
Those drawings show an Apple Pencil whose tip variously features a camera, interferometers, and what are just described as surface sensors.
Detail from the patent showing two possible configurations of an Apple Pencil-like device, one with interferometers in the tip, and one with a camera – image credit: Apple
This is not the first time that Apple has applied for a patent for an Apple Pencil that can detect texture. However, the previous example was about real-world applications instead of virtual reality, and was equally about color-detection as well as texture.
With this new patent, Apple appears to be focusing less on detection or measurement, and more on ways to make virtual reality feel more real. Sending such feedback through an Apple Pencil that the user holds would be less cumbersome than proposed Apple Gloves.
This patent is credited to five inventors, including Nicholas C. Soldner. His previous work for Apple includes multiple annual patent applications regarding ultrasonic sensors, and respiration detection in the Apple Vision Pro.
Note that even when a patent has been granted, there is no guarantee that Apple will ever release a product based on it. Patents can be filed on one part of long, ongoing research, and they can also be filed to establish prior art in case of future litigation.
Nonetheless, such a patent does at least represent a subject that Apple has devoted research time to. It’s also another indication that, contrary to rumors, the company is continuing to pursue its Apple Vision Pro and visionOS plans.
Stephan is the sports journalist for the Maple Grove Report.
