Sleeper Mobile Usability
Benchmarking the impact of "bloat" on core utility: A mixed-methods study.
Project Overview
Sleeper differentiates itself by positioning its app as a "digital playground" to connect friends over sports. This strategy relies heavily on integrated social features. However, recent aggressive expansion into sport betting, news posting, and more has introduced significant interface complexity.
While it may be profitable in the short term, this approach risks alienating core users who prioritize utility and ease of use.
Is this business model sustainable without compromising the user experience? Are these users against the new features a majority or a niche segment?
This study aims to benchmark the impact of these features on the app's core utility through a mixed-methods usability evaluation.
My Background In Fantasy Sports
I've played fantasy football for over a decade and have been the commissioner of my childhood league for almost just as long. Two seasons ago, I convinced my league to migrate from ESPN to Sleeper because I wanted to test out specific features that ESPN lacked.
Sleeper felt new and exciting, whereas ESPN had become stale and outdated.
As an active commissioner who likes testing new rules every season, I needed a platform that offered deep customization. As a frequent user, I want the app to be the best it can be.
Phase 1: Discovery & Definition
I conducted a "triangulation" of data sources to define the problem. This involved social listening to validate market sentiment, heuristic evaluation to audit the interface, and proto-persona development to target the right users before I started usability testing.
1. Social Listening
I utilized social listening on the r/SleeperApp subreddit. The goal was to move beyond my own bias and see if the "bloat" I felt was a widespread sentiment.
2. Heuristic Evaluation
I audited the mobile app using Nielsen’s 10 Usability Heuristics. This identified specific violations that served as the basis for my usability test script.
1. Navigation Identity Crisis
Violates: Consistency & Standards.
The app attempts to be a Chat App, a Fantasy Tool, and a Sportsbook simultaneously. High-value real estate (Bottom Nav) is occupied by low-utility pages like "Profile," while essential "Chats" are fragmented.
2. Task Friction
Violates: Flexibility & Efficiency.
When adding players, the default view is "Trending" rather than "Available." This forces an extra tap for every transaction, prioritizing discovery over the user's primary intent (utility).
3. "Mystery Meat" Navigation
Violates: Match Between System & Real World.
Terms like "Minis" are system-oriented jargon with no semantic meaning to the user. High-value tools (Research) are hidden under these ambiguous labels, killing discoverability.
4. Goal Interference
Violates: User Control & Freedom.
The "Picks" integration physically obstructs fantasy data. It is a secondary business goal (Monetization) that actively impedes the primary user goal (Team Management).
3. Proto-Personas
Based on the social listening data and my domain knowledge, I developed two proto-personas to guide recruitment for testing.
I expect to uncover other user segments during testing. But these steps helped guide my initial hypotheses below.
The aggressive integration of new features is creating friction that degrades the Social and Utility experience for core users.
Phase 2: Usability Testing
To validate the heuristic violations, I am currently conducting moderated remote usability testing through Zoom as well as in-person testing.
- Participants: N=6.
- Protocol: "Think Aloud" method.
Key Scenarios Tested
- The Utility Benchmark: "Find a Free Agent RB with the highest projected points." (Tests if gambling overlays create visual noise).
- The Dark Pattern Hunt: "Find the setting to hide the 'Picks' tab." (Tests discoverability/deceptive patterns).
- The "Recall" Test: Users close their eyes and describe the gambling features they just saw. (Tests Cognitive Load vs. Banner Blindness).
Phase 3: Programmatic Sentiment Analysis
Currently In Progress. Following the qualitative phase, I will perform a sentiment analysis to quantify broader user sentiment.
Using Python (PRAW library), I will scrape recent discussion threads to quantify the sentiment around specific features identified in Phase 2. This will allow me to validate if the friction points observed in 6 users are representative of the broader 10M+ user base.
Next Steps & Recommendation Strategy
The final goal is to provide Sleeper with a data-driven roadmap. This will likely focus on decoupling the "Betting" experience from the "Fantasy" experience through Context-Aware Navigation, ensuring that revenue goals do not cannibalize user retention.