ByteDance, Meta, and the battleground of "Pro"

It was just over a year ago that ByteDance revealed it bought VR headset maker Pico. At that moment, I think you would've been hard pressed to find a reporter or analyst who thought ByteDance had a good chance of seriously competing with Meta hardware any time soon. I didn't, and instead speculated on what Pico's catch-up software strategy could be.

Yet here we are. On Thursday, Pico announced the Pico 4, a consumer VR headset with specs that warrant headlines daring to ask if it might put the Quest 2 to shame.

Set to launch for consumers in Europe, South Korea, and Japan later this year, the Pico 4 will also manage to undercut the Quest 2's Euro pricing (though I have to imagine there was some celebratory fist pumping in the Pico offices when Meta raised the Quest 2's price). ByteDance even nabbed a plausibly strong exclusive game in the form of Ubisoft's Just Dance VR. The Just Dance franchise is not just popular, but so bizarrely enduring that Ubisoft saw fit to keep releasing annual installments on the original Nintendo Wii for a full decade. It's exactly the kind of game that could keep headsets from collecting dust.

Hardware, price, software strategy? So far, Pico at least deserves an A for effort on all three in this new consumer push.

But the 4 Pro could play a role in a bigger story

The least surprising thing out of the Pico 4 event, other than a tease of a Pico social VR platform, was word of an upcoming 4 Pro model, with built-in eye and face tracking sensors. Pico first put eye-tracking in a standalone headset before the ByteDance acquisition, so seeing a 4 Pro model that at least attempts to keep up with the marquee features Meta announced for Quest Pro last year also doesn't shock me.

When I examine the details, though, the Pico 4 Pro does intrigue me. With everything we've learned about the Quest Pro so far–and if you keep up with reliable leaks, it's quite a lot–it's hard to imagine that the Pico 4 Pro will outclass Meta's next headset on most, or even many dimensions. As far as polish is concerned, I would bet on Meta's headset coming out ahead. The question I have is whether the "Pro" market out there will think all the Quest Pro's nice-to-haves are really worth whatever potentially wallet-straining price Meta settles on.

Sure, ByteDance didn't sell as many headsets as it had hoped to in 2021, but we're only just beginning to see what the Pico might be able to achieve with that kind of backing on both the hardware and software fronts. The conspicuous number of west coast job listings sighted by Janko Roettgers earlier this year suggests it's only a matter of time until ByteDance makes a consumer VR push here in the US, but for now we only know that Pico plans to release the 4 Pro stateside. There's a good chance it'll only be available to developers and other professionals, granted. But if, amongst that set, the 4 Pro becomes a popular alternative to the Quest Pro, then it could be just what ByteDance needs before taking a bolder stand in this market.

Meta should pray ByteDance can't compete with a good enough "Pro" offering at a decent price yet, because if it can, well, it's only been about a year that Pico has had that rocket strapped to its back. Worrying about further gains on that front in addition to the looming threat of another possible "Pro" sounds like a bad time.

To the tune of Sussudio: "Rec Room Studio"

At the company's annual Rec Con last week, the biggest news that Rec Room slipped in amidst all the fun-having and stat-touting was the announcement of Rec Room Studio. Engine-maker Unity has partnered with Rec Room to let creators take advantage of some key Unity editor features and publish content straight from the editor into Rec Room.

This isn't quite jaw-dropping, but you're welcome to picture my eyes going very wide at the sight of Rec Room Studio. One day I'll try carve out a time to go deep on why I think any company or group with metaverse aims needs to learn some valuable lessons from the long history of game modding. For now, though, I'll spare you my complete syllabus on magnificent Doom levels and say this: if Rec Room and Unity weren't pursuing something like Studio, together or separately, it'd be a horrible missed opportunity.

For one, Epic Games is barreling toward its own take on this idea with the convergence of Fortnite and Unreal's editing tools, something Epic CEO Tim Sweeney says he hopes will lead to "a first-class outlet" to sell games to consumers. Having overseen the glory days of Unreal Tournament, he of the all-powerful plasma ball is no stranger to what can happen when game companies give people the right creation tools and get out of the way. Then again, it can be hard to believe that a company as acquisitive as Epic has been will be a better long-term storefront steward than Apple or Google has been.

Of course, Rec Room is in the business of keeping creators generating revenue on its platform too, but there's a potential path off-platform that makes Rec Room Studio appealing in concept.

Imagine a young creator in middle school who can hardly remember a time before Rec Room (if I just made you feel old, I'm sorry). This creator is used to using the in-app tools, but moving to Rec Room Studio doesn't just open up possibilities for them thanks to the Unity editor–it also pulls in the wealth of existing Unity documentation and tutorials that might come in handy as they're learning. Create a game with Rec Room Studio and those project files might only be a few steps away from being a viable standalone Unity-powered game outside of Rec Room.

With Epic holding both halves of the puzzle between Fortnite and Unreal, the likelihood of developers struggling with a Roblox-esque lock-in there seems obvious. I see more of a chance that Rec Room Studio could be a more freeing pathway into development, but the devil will be in the details. However it shakes out, I suspect we'll see the average age of developers grousing online about Unity workflows and bugs fall by a few notches.

Brainy body tracking and a little bit of Bonelab

The latest bit of Meta VR research to make waves is a paper with the unassuming name "QuestSim." A goofier but more fitting title might be "Body Tracker Companies Have Near-Death Experience With Machine Learning Black Magic."

Jokes aside, the upshot of this paper is that Meta is pretty close, evidently closer than a lot of observers thought possible at this time, to generating very accurate poses for a user's body using just from plugging the 3D positions of a headset and controllers in each hand into a reinforcement learning model and physics simulation. Caveats aside, the results are just plain noteworthy; when you've seen enough bad body tracking and pose estimation, the moments in the research video where the simulated body post matches the real perfectly are striking.

Okay, but about those caveats: along with a few edge cases like dancing and boxing that make the simulated body look like a floppy ragdoll, there's also an air of drunkenness around some of the straightforward walking and running motions that shouldn't be there. Combine that with latency or, as David Heaney pointed out, even a minor mismatch with your real hand positions, and a first-person experience with QuestSim might be a one-way ticket to Nauseaville.

You know that part in every paper where the authors gesture at future avenues of research? It just so happens that I read this paper on the same day that VR developers Stress Level Zero gave a first look at new features in Bonelab, their upcoming VR sandbox game, and both the paper and gameplay video address the same idea of mapping a user's tracked motions to avatars with vastly different proportions. Stress Level Zero appears to be using a more traditional inverse kinematics (IK) model to pose an avatar's joints, but what they've shown of their results in a game shipping this week are impressive.

It's not time to claim the death of positional tracking accessories yet, but if machine learning and/or widely used IK models keep improving at this rate, trackers might become a harder sell than they already are. At the same time, it'll also be harder to justify packing additional sensors into headsets for the purposes of tracking if pose estimation can get most of the job done. The fewer cameras tracking your body and your surroundings, the better?

Bringing a different kind of "flat" to VR

If you're reading this, thank you! Moving forward, my plan for each Tuesday is to tackle three or four items at the level of detail above, with free subscribers guaranteed to get at least one item in full.

Further down the newsletter page I'll direct you to other stories of interest, but I want to reserve this place in each issue for a spotlight on something I saw each week that just excited me about immersive tech or the communities surrounding it. So queue up the fanfare…

It's been a great month to follow what's happening in the Flat2VR mod community and games Twitter more generally, thanks in large part to the emergence of Trombone Champ. Developed by Brooklyn-based Holy Wow, Trombone Champ is an absurd rhythm game where honking out of tune is most of the fun. Leave it to prolific modder Raicuparta to get VR trombone gameplay within days of Holy Wow's game going viral–if you want to check out another, less silly mod, I can highly recommend Raicuparta's VR conversion of Outer Wilds, which is far more comfortable and immersive than I would expect an unofficial VR mode to be.

As of this morning, the "BaboonVR" mod for Trombone Champ is available in an early state to Raicuparta's Patreon backers or for $2 on itch.io. It'll become free and open source later on.

More stories:

• Today, Epic Games and Autodesk announced a new strategic partnership that will see Autodesk Revit subscribers receive free access to Epic-owned Twinmotion. If, say, a VR walkthrough of a design built with Revit and brought into Twinmotion isn't quite flexible enough, Epic promises it's a short leap from there to working with Unreal and its long list of native tools, asset libraries, and outside integrations.
• Meta released the program for its upcoming Connect conference on October 11. Expect to see thoughts on the keynote and more here in the Virtual Vector newsletter afterwards, but watch this space for more details on other ways I might cover the event.
• A new software update for Quest adds a parental option to remove developer mode, which blocks the ability to sideload, and adds some recording options that should be familiar to SideQuest users. [Scott Hayden / Road to VR]
• Niantic has officially launched its Lightship visual positioning system as part of the WebAR tools provided by 8th Wall, which it only acquired about six months ago. That's seems like a pretty fast turnaround to me. [Niantic press release]
• HaptX, the Redmond, Washington-headquartered VR haptics glove company, raised $23 million in a round led by AIS Global and Crescent Cove Advisors. No update on the status of HaptX's dispute with Meta over microfluidics-driven haptic tech. [Kurt Schlosser / GeekWire]
• Lamina1, the new metaverse/crypto venture with author Neal Stephenson serving as co-founder, released its first whitepaper through its Discord server (though, fun as Discord is, you can also find it here). It's not deeply technical, but you might want a guide to blockchain buzzwords and a glossary of some Stephenson-isms at the ready. If the paper reads a bit like sci-fi, well, that's because at this point it kind of is.
• "How to Build a Metaverse" is a new four-part podcast miniseries covering the creation and heyday of Second Life. If you were ever active on Second Life yourself or have followed what Philip Rosedale has been up to recently, much of what's covered in the first episode will be familiar to you, but I'm looking forward to seeing where this series goes. [Annie Minoff / The Wall Street Journal]