4.5 stars, 200 reviews and still having problems? A common occurrence in the Shopify App Store.
What we look for when choosing Shopify apps
Philipp Scambor
Director E-Commerce & Co-Founder @ Digit-One
We regularly evaluate apps from the Shopify App Store on behalf of our customers and, through explicit guidelines and experience, have developed criteria against which we assess them. One pattern keeps cropping up, and it is particularly important for less experienced retailers to be aware of it.
The Support Rating Trick
An app has 4.5 stars, 200 reviews, and sounds solid at first glance. Installed, set up, and then: problems. An error here, an incompatibility there. Support responds promptly, kindly, and resolves the issue. The merchant is relieved and leaves an enthusiastic review: “Great support! Helped me quickly solve the problem.”
What seems positive at first glance is, in truth, a warning signal.
The pattern behind this is as follows: The app is unstable or difficult to use, problems arise, and the support (often in a low-wage country with correspondingly low costs) is well-staffed and responds quickly. The problem is resolved, and the relieved merchant is asked for a review at this opportunity. Cycle begins.
Whether this happens intentionally or is simply the result of many real problems cannot be judged from the outside. But both options are bad for a merchant: either review manipulation or an app with too many bugs.
Why the Pattern Works So Well
Reviews are generally a good signal, but support ratings are different from product reviews. An app that works reliably rarely receives a rating. No one writes, “Everything went as expected. 5 stars.” An app that causes problems and then resolves them creates the perfect situation for a review request with a merchant who is just relieved and grateful.
The simplest rule we derive from this: A good app should not need support. Not because support is generally bad, but because an app that simply works does not provide any opportunity to use it.
Context is Everything
This does not mean that all offshore apps or apps with good support should automatically be avoided. The critical factor is the risk in the respective usage context.
For a small shop with manageable volume, friendly, available support can indeed be practical. An app that fundamentally does what you need and where problems are quickly resolved is a valid choice in this context, especially if the alternative is an expensive custom development.
In a larger shop with high volume or a critical function, the equation changes. A checkout app that occasionally has problems can, in the worst case, cost orders. An inventory sync app with bugs leads to overselling. Here, an app should not need any support at all, and if an app developer has to actively take technical steps to make the app work in a specific setup, that is a red flag and not a service.
Known Providers First, but Still Evaluate
For established use cases like reviews, search, or loyalty programs, there is often a market-leading app from a known provider. Choosing this is generally the safer route. However, an evaluation is always worthwhile, if only because revenue-dependent licensing costs for larger shops can tip the cost-benefit equation.
Read Reviews with a New Perspective
Not just the average rating, but the content. Do support experiences dominate the picture? Are the reviews clustered in time (often a sign after a launch with active review outreach)? Similar phrasing in many reviews?
Check Changelog and Update Frequency
Does the app have a changelog, and does it describe real changes or just vague “bug fixes”? When was the last update? An app without active maintenance is a risk in a rapidly changing ecosystem like Shopify, as API changes can break apps overnight.
Testability as a Dealbreaker
Can the app be fully tested on a development store or in a trial phase? If so, we actively test it, not just the happy path, but edge cases and behavior in the real setup. If not, or only to a limited extent, other options come first.
Why the Quality of the Shopify App Store is Becoming a Bigger Issue
The quality problem in the Shopify App Store is not new, but it is worsening. The reason is simple: AI lowers the barrier to app development to almost zero. A developer can build an app over a weekend that appears convincing in the demo and covers the happy path cleanly. The problems only show up under real conditions, such as edge cases, under load, during interactions with other apps, or in unusual setups.
The real risk lies in the code, which in the end, no one really understands (sometimes not even the developer themselves). “Vibe-coded” apps run as long as everything fits. The next Shopify API breaking change or the next unexpected setup will reveal whether someone is behind it who can maintain the code or whether it is a ghost app: listed in the store but effectively abandoned.
Conclusion: When to Use an App, When to Go Custom?
Evaluating an app before considering a custom development is almost always sensible. There are mature apps that reflect years of development work and work reliably for most setups. Reinventing the wheel rarely makes sense.
At a certain shop size and criticality of the feature, the equation shifts. If an app is not fully testable, if the review pattern sends warning signals, if the developer is unknown and support is the most common review content, then the risk outweighs the benefits. In these cases, a custom solution, even if it seems initially more expensive, is often the cheaper option in the long run.
And if you’re unsure: As a Shopify agency, we guide our clients precisely in these decisions. An assessment from someone who knows both (the app store and the possibilities of custom development) is not a luxury expense for critical features.