BT

Facilitating the Spread of Knowledge and Innovation in Professional Software Development

Write for InfoQ

Topics

Choose your language

InfoQ Homepage Presentations From VR to Flat Screens: Bridging the Input and Immersion Gap

From VR to Flat Screens: Bridging the Input and Immersion Gap

39:18

Summary

Dany Lepage discusses the architectural journey of porting a hit VR title to seven non-VR platforms. He explains how his team solved the challenges of cross-progression, diverse input paradigms, and maintaining release velocity across Steam, iOS, and PlayStation. Beyond the tech, he shares candid lessons on the "product fit" gap when translating immersive social presence to 2D screens.

Bio

Dany Lepage has been Technical Director and VP of Technology at Lucky VR for over 3 years. Dany started his career in microelectronics at Matrox Graphics before moving to Nvidia. He then spent the next 15 years building video games at Ubisoft and then 5 years in location based entertainment at Framestore and Solotech.

About the conference

Software is changing the world. QCon San Francisco empowers software development by facilitating the spread of knowledge and innovation in the developer community. A practitioner-driven conference, QCon is designed for technical team leads, architects, engineering directors, and project managers who influence innovation in their teams.

Transcript

Dany Lepage: At Meta Connect, there was a great talk, part of the technical keynote, where Richard Newcombe talked about the future, talked about how things were going to evolve, and how it would naturally go toward having always wearable. We were talking about the glasses and also about VR in general. We talked about the digital first wave and then talked about the second wave that is much more natural in terms of interaction, that doesn't distract you from the environment around. Very interesting, big presentation. That's essentially what created the success for Lucky VR.

Several years ago, we did a fantastic game that became a VR phenomenon. I'm going to explain to you why in a moment. We're talking about a relatively small market here, so the revenue performance is fairly significant.

The level of presence you have when you play poker, for example, is just unmatched. Because you can imagine that there's people around the table. It's like the perfect setting. I don't need to walk around. I don't need to be in an environment where I need to go everywhere. I just sit around the table. It's perfect for the current limitation of VR. I'm just going to show you very quickly what it is. That's going to give you an example of the type of immersion. I didn't get a great hand, so it's probably better to move on.

The market reality today, it's a great adventure, but the market for Quest for VR device, it's about like 20 million units. It's not a bad business, but 20 million, it's less than the PlayStation 5, which is close to 100 million, and even more so than mobile or PC. That's a problem. When Quest 3 and 3S came out, that felt a bit incremental from the Quest 2. Quest 2 did reach a really high selling, and we thought that the Quest 3, 3S were going to raise that bar. It didn't quite happen. That didn't get us the expansion we expected. Of course, just like a business, we want to make money, we're greedy. We want to get into a business where there's going to be more opportunities for growth.

Essentially, this story today, it's going to be about how we have taken a VR game and we brought it to previous devices. We're talking about phones, about PCs, even the PlayStation 5, so many devices. This is going to be a great story giving you an idea of what we did over the years. That expansion started in 2023 with the PlayStation VR2, that was the easiest. There were a few things we needed to do for that platform specifically, because on the Quest, we were using the avatar system from the Meta ecosystem. Then when we moved to the PlayStation, it wasn't possible to use those avatars. There was a need to use a different type of avatar.

Then we moved to Steam Non-VR, which was a big endeavor, because when you're not in VR, that means that you need an animation system, you need a state machine, and you need to evolve your game differently than using mocap. Then we moved to Android Mobile from there, PlayStation 5, which was a big lift, iOS, and recently even the Samsung Galaxy XR. Every platform that I'm talking about is cross-progression, cross-play. Even if they have a different avatar system, even if they are different, they have different characteristics, you can play against everybody. Anybody on any platform, you can play at any time.

Roadmap

I'm going to talk about what went right and what didn't. Curiously enough, in that crazy adventure, the technical aspect went very well. I would say that from an engineering standpoint, we did hit all the targets. The game did run on all the platforms. We were successful at adapting the game. The gameplay was there. It was fairly decent. What didn't go so well is the product fit. It was much more difficult than we thought. We never expected that we would have the same success with the game on mobile that we had in VR in terms of attach rate, but it was below expectations. That's also something I'm going to talk about towards the end. Also about future opportunities, what we missed, maybe we should have done it differently.

Background

My name's Dany Lepage. I've been in that business for a very long time. I started my career in microelectronics with NVIDIA, Matrox. Then worked at Ubisoft for 12 years. I worked on Splinter Cell, Rainbow Six, Far Cry, Assassin's Creed, Prince of Persia even. Then started my own company on mobile. I also worked in location-based entertainment at Framestore, at Solotech. For almost more than three years and a half I've been working to build that metaverse, that platform expansion at Lucky.

Part 1: Architecture and Technical Pipeline

I'm going to start with the architecture and the technical pipeline. I've chosen four platforms that are fairly different from another. Android and iOS, I bundled together. It's not exactly the same thing. Some of you may have done development for iOS, Android. There are differences, but less interesting. Meta Quest, PC Non-VR, Android/iOS, and PlayStation 5. Right now, we have two avatar systems. We have the Meta Avatar system that is actually pretty amazing.

One problem though, it doesn't work on iOS, it doesn't work on PlayStation 5. How do you expand? We needed to build our own avatar system, creating tons of problems, because like I mentioned, there's cross-play. I think you can probably already imagine how difficult and how challenging that is. The control scheme, very different as well. We're talking about motion capture with the Quest. We're talking about a mouse-keyboard on PC. We're talking about a gamepad on PlayStation 5. Obviously touch on Android, iOS. You have different stores. That's pretty much the same thing as differentiating between accessing Google Play, the App Store. Now you move into Steam, you move into PSN. It's a bit different, but same kind of principle.

Finally, there's different limitation on different platform. I'm going to maybe talk a bit about those limitations, but the Quest 2 is by far the device that has the most performance problems. You wouldn't think so. You'd think, it's a gaming machine. Please remember that we're running at a minimum of 72 FPS, and ideally at 90 FPS. Dropping a frame is much more damaging to the experience compared to mobile, because it creates a bit of motion sickness. We need to be very disciplined. 90 seconds goes fast, means that to render your frame, you have 11 milliseconds. Not that much.

Technical stack, also different from maybe other projects that are outside gaming, but also a few things that are very unique. We use the Unity game engine for this product, specifically Photon for network synchronization. AWS for the backend, for non-latency critical storage async. Noesis for the GUI. It's also different because you can imagine in VR, you have UI that is located in 3D space. It does require things that are a little different. For voice chat, we use Vivox. We have ToxMod for age moderation, for example. We're actually one of the first customer to use ToxMod. Just by the voice, we're able to detect that user has maybe a 99% chance of being underage.

Then our customer support department can act very quickly to ban the user. We're also using Jenkins, and very surprisingly, GitHub. Video games, GitHub is not being used that much, has severe limitations, so generally like Perforce is preferred. Specifically for this project, we did use GitHub.

I'm going to talk about some of the challenges one by one. I'm going to start with one of the most interesting one, avatars. In 2023, we had what we call Meta Avatar Style 1. Meta Avatar Style 1 didn't have legs. They were very well done. They were available on PC Steam, VR, non-VR, Meta, obviously. That was the basic. Then we had a lot of discussion with Meta about, do you want it to be the provider of the digital identity of the future or not?

At some point it was yes, and at some point it wasn't. There was a lot of evolution. At some point we just gave up, and we decided to build our own system because it turned out it was the only way to expand to more platforms. We did build the Lucky Avatars with an aesthetic that was comparable to what they did at Meta. That came out for the first time on Android, if I remember correctly, in April 2024.

Then we moved that across all the other platforms. We're actually thinking about migrating. That would have been so much easier. Migrating the Meta Avatars to Lucky Avatars would have been way simpler. Unfortunately, or fortunately, Meta came up with a great solution in 2025, the Meta Avatar Style 2 with legs. Those look very good. After a long discussion internally, we felt that it was going to put us at a competitive disadvantage if we were going to migrate to Lucky across the board. We ended up saying, we're going to be stuck with the Meta Style 2 avatars on the Meta platform and everything else will have Lucky.

Talking about the challenges, let's talk about the first one. Very different one. Imagine that you have two digital identities now. Imagine that you're a player on Quest, how do I look like for people on mobile? Maybe I haven't played on mobile. I don't know what I look like. It's a different avatar system. Very difficult, very challenging. Surprisingly, the way we've dealt with the problem was actually not handling the problem. We just decided not to talk too much about it. People were able to go on the other platforms and they could build their second digital identity.

Maybe it was the same, maybe it wasn't, but it was up to them. We just said, we're not going to try to solve the problem because that was going to be quite complicated trying to match. Do we support the exact same skin color? Do we support the same exact body type? Do we support the exact same thing? People, when you start getting into that challenge, it was likely going to be a mess. One area though where it was a bit of a mess, where I think it's something maybe you don't expect with an avatar system, but DEI. We're talking about diversity, equity, inclusion. Why am I talking about this in tech? Because it matters. If I don't support DEI, I have male, female body types. That's it. Simplifies the problem.

If I have a DEI compliant system, I need body type, so, much more diversity. That means a different mapping system. You need to build a system that is much more resilient and much more able to adapt to different variants. Had a big impact for us. Eventually we pulled that out. Probably something you don't know, but even the Meta system is DEI compliant. There's no concept of binary. Quite interesting as well. Then there was also the performance problem. Obviously, when you think about Meta, they didn't have in mind the idea that somebody would build something that crazy. They didn't make their avatar system to be interoperable with other avatar systems. It was very difficult to essentially be able to properly communicate, because on one platform you have an avatar system, and on the other, you have another avatar system and you need to communicate. I'm saying here, choose your poison.

For example, you're on the Meta platform, I have my three points of control. That's three quaternions. I can transfer those quaternions very efficiently, but then I need to run a full body IK for every character in the universe if I do it. Quite expensive on the processing. Even more so if I do the exact opposite because the Quest is very limited from a performance standpoint. On the other end, I can decide, I'm just going to process that IK and I'm going to transfer all the bones. I can use compression, but that's more expensive on my network cost. We ended up choosing that solution. We ended up paying the price in terms of bandwidth. It did bloat our operation costs with Photon, but it was the only way we found because Meta didn't necessarily build the exact system that was going to be ideal for us to fully manage cross-compatibility between two avatar systems.

Another one, it's also very obvious. When you were building apparels, apparel is a big part. In free-to-play games, you're selling apparels to your players. Now we need to adjust them for two systems, Lucky Avatars, Meta Avatars. Sometimes it doesn't fit, avatars are a bit different. You don't start from scratch, but it needs adjustment. It's not fully automatic. This is quite a lot of work. You can imagine how it bloats in terms of QA, when we release a new version, every month, you're adding new props. Now it needs to be tested with two systems. Also, as you can imagine as well, when you do an upgrade, you update the different SDKs, can be quite challenging.

Now I'm going to talk a little bit about the input system. Somewhat less challenging or more straightforward. Again, I talk a little bit about the Quest, mocap, PC, mouse-keyboard, PlayStation 5, gamepad, mobile, touchscreen. The level of interaction is quite different here. I showed you the video earlier where we have a Quest player, which was actually me using my own body to move the cards around, play with the chips. That's great. That's how I would play poker myself.

If I go to a WSOP tournament and I'm playing with my chips, I'm a bit nervous or I do whatever, that's very close to real life. I actually move my body the same way I would at a real poker tournament. Another platform, not quite the same. Mouse-keyboard, gamepad, very different mapping. I'm going to talk a little bit about it. Create an indirection as well. The microphone, which is so important on the Quest to create presence, people on mobile won't use it. Quite a significant challenge. I'm going to show you here one example that's going to help you understand. This is on PS5, something I captured. Just the menu here that you see on the left, being able to navigate through it.

As you imagine, like on console, there's a big drawback. There's no pointing device on a gamepad. People hate that they hate to move a virtual pointer with a thumbstick. Obviously, the classic technique, I'm just going to show the video again, is to use indexing with the D-pad. Very typical, again, for PS5 games out there. The D-pad, if you look at the pictures at the bottom, that's a D-pad. That means that you can navigate and move from one element that you can interact with in the menu to another. Very simple, but it still means that every time we design something, we design a new game, we have to think about all those platforms.

Another thing that's not necessarily obvious here and that's also different from most apps that have an iOS component, iOS platform, and Android platform, and people use one or the other. This is not true for metaverse application. The core concept of metaverse is people use multiple platforms to access your universe. That also make analytics much more complicated. What we had to do is build a custom analytics system where we need to know what is the first platform that a player was acquired with.

For example, very typically, I'm giving an example here, you begin your tournament in VR. Amazing at presence, you have a great time. Dinner time. How do I complete my tournament? Maybe I have enough or I have to do something else. I'm going to put my headset away, switch to my phone, and then finish my tournament there. Very common use case. Great, but how do I know if that player is a new player, that player is an existing player? You need to have some subtleties there. It's going to become even more obvious when I'm going to show you the product stats and why this is so important.

The release management at scale, that is also something we did very well, something I'm really proud of. I think the teams did pull that off very well. We have a relatively simple by the book process. The person in charge, the QA director, it's an interesting story. The way she works, she's very disciplined. It's essentially operating like a car factory, where you get the version that is instabilized, it goes through the process, and then at the end, we release it. We have three versions active at the same time. We have development, we have UAT, probably last one, two weeks, and then platform certification, and then live deployment. We have a live team that works on the live platform to create new tournaments, essentially operate the live games. We have the developer working, and we have a team in between that is working on stabilizing the next version.

There are generally three active versions in our company at all times, plus, obviously, the roadmap. That's fairly interesting to manage, but it works very well. It's essentially a factory when it gets to that point. To make it happen, I'm going to talk about it later. This is a very simple system, it's in Jenkins, but also very important, being able to output directly to the channels of the different platforms. When the QA team is getting in the morning, and we're ready to start the release process, very important, I have the version I need to test ready on my Quest. It's ready on TestFlight. I can access it on Google. I can access it on my PS5. I can access it on Steam, on the release channel. It becomes all available at the same time, automatically distributed. Extremely important. That made the biggest difference in our success. It's very disciplined.

Also, we are ready for worst case scenario. It's also something pretty obvious. There, there's going to be a bit of a compromise. We do have a plan when things fail. You may be aware that some of those platforms that I talked about may not be able to certify your games within one hour. Like when we're talking about Steam, we're talking about Quest, you auto-certify. If you make a mistake, you can fix it essentially immediately. Not happening with Sony. Not happening with Apple all the time. Sometimes there's more people on vacation. They need to go through an approval process.

If you make a mistake, you're stuck in a situation where you may be able to fix it on Quest, but it's going to take a while before it's fixed on the other platform. We have a backdoor where we break cross-play if we need to. Obviously, nobody wants that to happen. It's better than shutting down your game. Quite an important process here. Just to finish over the release process. It's a process that pushes discipline over agility. Again, it's very much like a car factory. It was a big change for the team where initially release was operated more like development. Very different. I actually had to do it myself. I played the role of release manager for a few months, just to ingrain the change of patterns in the team and that they could understand that, no, this is not development. You don't do things in order. It's more like launching a rocket where there's a succession of steps that you do.

Then when you fail, you cancel the launch. At development it's non-linear. There's PRs, submits going from everywhere, out of order. You don't do that for a release process. You go through steps. Then when it fails, go back. Very organized. Very linear. Then we just started to get very good at it because we have a great QA director. We start iterating. We optimize the process. We get faster at it. It's just like a car factory. Just like Toyota was doing in the '80s. It's just run, optimize, run, optimize. Faster, faster. Always very disciplined. Made a big difference as well. Today we're able to release, most of the time, on all those platforms very efficiently. That's pretty much a technical success.

We have seven platforms releasing simultaneously at the moment, so every three weeks. It's also impressive. Different clients. We have two avatar systems. At some point, we even had three. We had Ready Player Me, Meta Quest, Lucky Avatars. We streamlined this to two. Now we have four input paradigms that are very different. The thing that we're probably the most proud of is even if it was difficult, we're at the point where feature velocity is probably the same as it was when we were in VR. Developers are always complaining that it could be better, but it got to a point where we did improve our efficiency to be able to compensate for those extra platforms.

Essentially, it was a big cost, big investment. Those platforms are not adding to the weight. We didn't lose our velocity. We didn't lose our ability to output new features. It's a big change, obviously. You can imagine that people that are developing for, let's say, a new slot game, they're developing Craps, they're developing something brand new. They need to make sure that they support all those platforms, all those stores, all those input schemes. They're able to pull that off. It was pretty successful.

Part 2: Product and Business (Multi-Platform Expansion)

Now we're going to talk about the part that didn't go as well. I'm going to go back to why we did this, because this is more important than the tech side. Why did we expand? Again, there's 20 million units on Quest. We wanted to move to markets that are bigger because there's more possibilities. Of course, we're greedy. We want to succeed there. The second one is the player liquidity. We thought that by having so many players in VR, player liquidity is very important in those type of games. Any game that is multiplayer, essentially. People want to be part of a big ecosystem. We thought that by having that player liquidity base and expanding to another platform, that would give us a competitive advantage.

The third one was cost optimization, which ended up being true, like I mentioned, where right now releasing on multiple platforms is far from a disaster. Those extra platforms, let's say with all the optimization, maybe they cost 10% more than if it would be single platform. So many more opportunities for revenues for an extra 10% cost. Obviously, business case was quite interesting. Again, we never thought it was going to be a billion-dollar business on mobile. We knew that this game was a bit hardcore, not exactly a perfect match for the platform. Still, the competition was very strong. That's a drawback of the metaverse approach where you develop a single game system, very backend authoritative. You try to make it work on every platform, but the fit is different.

For example, in VR, celebration is a huge deal. I'm winning, there can be fireworks. On mobile, it's a bit boring. I would probably want to move to the next hand much more quickly, but I can't because I have VR players against mobile players. I'm stuck in some kind of compromise from a product standpoint. That didn't help as well. There are advantages, but there's also disadvantages. There's also an inferior product fit that I'm going to talk about. It's probably where we made a bit of a mistake. That's information about our game at the moment.

As you can see, I'm a huge fan of day 1. Huge fan of day 1 retention for market fit. On Meta Quest, it's very high. It's like 40%. This is pretty good. When we drop to PC Steam, it gets to 30%. When we get to PS5, mobile, it's 25%, 26%. Not great. Another example here, we develop essentially on mobile, the perfect companion app. I mentioned the use case where you have somebody who started experiencing Vegas Infinite on VR and then is moving to mobile. It's getting better actually. That 54%, it's higher now. Those guys, they love it. At this point, like removing mobile would be a huge loss for VR players. It's 54%.

Then VR, very good as well. That's for main acquisition channel. Mobile first, 25%. That's where it hurts, because on mobile, you're in the red ocean. You need to do user acquisition, and you can survive with those stats. Of course, there's more than that. Monetization is actually decent, but you won't survive. You won't be able to do user acquisition if your KPIs are not good enough. You just have no existence whatsoever on those platforms. That was a bit of a surprise. We thought there was going to be an easier way to find a niche, but it didn't quite happen. Let me explain the mistake we made in my opinion. This is a slide that I took from a presentation from 2021. It was made by my CTO, Dariush. I felt that this slide represents what makes Vegas Infinite.

At the time, it was called PokerStars VR actually. What is making that game great? It's focused on recreating small group settings, presence and fidelity of interaction extremely important. That's the second one that we're going to talk the most about. This game is not just about playing poker. It's about the social experience. It's about using props, about talking to people, making friends. This is social VR. We didn't succeed at creating the same kind of experience on non-VR. We were not able to convey that experience. Maybe that's actually not what most people on mobile are looking for, but we should have tried. We didn't.

The social experience that you have on the non-VR platform is not the same. You cannot use your body, "Hey, I won". It's not the same thing pushing a button. You don't feel it the same way. Playing with your chips, who wants to have a button to play with my chip? The fun is actually picking up the chip. If you cannot use your own body, it doesn't have that much value. I'm just going to show you another example here. A great interaction. How do you do that on mobile? How do you do that on a PC with a mouse-keyboard? That's actually fun. People do it. If you go to a poker tournament, people play with their chips. VR, it's actually fun, just like in real life. Not on the other platforms.

We thought initially that immersion was important. We thought we were going to have at least some kind of competitive advantage because we had 3D avatars. We had better looking games than the competitors. The other games were in 2D. We thought that that means something. It's probably not going to be the perfect fit, but it was going to be fairly unique. We found that input is the big thing. Being able to express yourself, if you're social on a platform, that was the key strength for VR games. You need to be able to carry over. There's a game that is doing extremely well on the Quest at the moment called GOLF+. It was discussed a lot at Meta Connect actually. Look at the guy there. You actually hold the controller and you do a motion. You've seen golf games on mobile where you have some kind of wheel that moves around. If you tap it exactly at the right spot, you're going to drive straight. It's not the same thing.

More than 50% of their users on GOLF+ are real golf players. Why would a real golf player want to click on a mobile screen? Maybe not. Maybe they're not going to port it. Does it even make sense to adapt the VR games to the first wave of digital devices? Was that a doomed effort? Was there a way? It worked technically, but maybe that was a stupid adventure. Maybe it was never going to work. Let me talk about a few things that I feel could have made the difference here. I'm going to talk about two examples.

The first one is, how can we express ourselves? How can we do it in a way where I win? I want to express myself. Obviously, it's not going to be the same thing. I'm not going to lie to you. There's no way it's going to be as expressive. The amount of information that you're transferring is too limited. The first one is Governor of Poker. They have a contextual menu that appears at the bottom. Every time you do something, there's choices how to express yourself. We could have done it with animation. We could have done it in a way where, "Oh, that hurts".

Then proper body animation, and then convey something. We're at the point where even the VR players, they actually hate to play with mobile players because the mobile players don't do anything. They're just not expressive. They're sitting at the table and we play idle animation, like an NPC in a typical video game. Not super exciting to be sitting next to one of that person, because when you play in VR, you want a social experience. It wasn't great. That's actually a game that is actually part of our portfolio. We should probably have taken a look earlier. It's the PokerStars Real Money app. They have a prop system that essentially when you use props, you use them on other players. That's how you express yourself.

For example, here, there's an example, there's tissue. If somebody is really unlucky, like he bet an all-in or something and then he loses, I can just click quickly on the tissue and click on him, and then he's going to send the box of tissue. There's going to be a slow animation. That was brilliant because you use a prop system that would look very good on 3D to express yourself. Those are ideas. It's never going to be the same as VR. It's never going to be the same as being there. That gives you an idea of where it could be. An important lesson to think about this adventure. It's not over. We're always trying to improve.

Key Learnings and Conclusions

Some of the key learnings from the experience is, technical execution, I think that's probably true in most companies where you can get great people and great people will solve difficult problems. They will find a way to get results. When you get to product fit, it's much more difficult. Operation went better than expected. Cost is lower than expected. The cost of having and carrying all those platforms is lower than expected. The market performance is under expectation. We did develop like a jack of all trades. That's a problem that a lot of games have when they try to move from VR to other platforms. Even up to a point, Meta Horizon. It's clearly an experience that can be pretty good in VR, but when it moves to mobile, even Rec Room, which was probably the biggest success from a game that went from VR to mobile, average.

Even today, most of the revenues are still in VR. Not obvious that this is going to be a huge success. Technical problems were solved, but gameplay, we still have a long way to go. I still think we can adapt the game and maybe find our niche. It was definitely harder than I thought. I didn't talk about a few technical subjects that I could go into. Maybe that's for another discussion about app size, how you optimize. There are some great engineers here who worked on addressable, being able to split the app. On some platforms, the game is more like even 13 gig or 14 gig. When you move to iOS, you move to Android, you need to be able to swap around the packs. Those problems can be solved. Your engineers will be able to solve problems. Product problems are harder.

 

See more presentations with transcripts

 

Recorded at:

Apr 16, 2026

BT