Are Skillshare Search Results Broken?

Are Skillshare Search Results Broken?

This post was most recently updated on October 15th, 2018

Skillshare made a significant tweak to their search algorithm recently. We believe the change was made around the beginning of June 2018.

The first reports of the change in a negative sense came on June 3rd, by David who reported in our Facebook discussion group that his premium minutes had “plummeted” since 1st June.

plummet

Interestingly, I’d made a post on June 1st asking if anyone had seen their premium minutes increase over previous days. I proposed that a new Skillshare promotion was driving more signups and premium minutes. Not many people agreed that they were seeing more minutes, and it was mentioned that the new promotion may be targetted to specific demographics. In hindsight, perhaps this was an early indication of the SEO change, because, unlike David, I’ve seen my premium minutes increase.

 

Changes Reported By Skillshare Teachers…

So what information do we have? Well, David said that some of his most popular classes can no longer be found by searching for them (“many of my bestselling courses that had previously appeared top, or near the top, for their particular keyword phrases have suddenly disappeared without a trace – i.e. I am unable to find them with a direct search for the title of the course“).

Mark replied and agreed that he was seeing the same thing, “our courses have disappeared out of search“.

Sergey also agreed, “Yes I seen this happened as well all my best selling classes no longer showing up only newer classes are”.

Linday saw the same thing… “If you type in graphic design you do not come to one of my classes until 50+ rows. I used to have two in the top 10 (older class) so it looks to be an algorithm change that pushes older class lower (except their top teacher and their produced classes).”

David gave further details

“Up until the 1st June my Beginner’s Guide to Merch By Amazon course was top of the search results (as it has been for over a year, ever since I released it) and was getting new students every day. Now, the course is ranked 224 for the key search term, Merch By Amazon, and is actually ranked directly below a class with ZERO students called “Hand Made Creations Part 3 Amazing Fish”.

 

Skillshare’s Response…

Subsequently, Mark made a new post, proposing that Skillshare search was “broken” and including a response from a Skillshare community manager…

Hi Mark,

First, I’d like to confirm that I’ve double checked and your class is open and viewable to students.

We are constantly making updates to our search algorithm, which may affect where your class appears on Skillshare. This algorithm is machine-learning based and takes into consideration a variety of factors, such as current trending popularity, course topic, category, and length.

Let me know if you have any additional questions about this.”

In response, I noted that, “because of their business model, new classes are constantly required by Skillshare. This search change seems to be weighted towards new (or popular) classes, even to the extent of sometimes beating classes with titles matching the search terms. It’s interesting to note that, in their reply, Skillshare actually listed the factor “popularity” before “course topic” for determining result positions!”

 

David replied, agreeing with Mark…

“I totally agree with Mark. The search is broken.

I totally understand giving priority to new classes (and to a lesser extent “popular classes”), BUT at the moment it appears the topic does not even feature in the algorithm, which is ridiculous.

For example, I could sort of understand if my Merch By Amazon class was listed last of all of the Merch By Amazon classes because it is the oldest (although I would not agree with that policy). However, coming below classes that have ZERO students and nothing in the slightest to do with the search query (apart from 4 letters in a different word within the title), just because the class is new, makes the Search function a complete farce.”

 

The Facts…

So what do we have here?

Firstly, there was a definite search algorithm change recently at Skillshare.

Secondly, that algorithm change reduced the visibility of some, but not all, classes which were previously ranking well.

Thirdly, while the age of a class may play a role in the demotion of some classes, and it does make sense that Skillshare, who charge members monthly, should promote new classes over old ones, age is not the only factor, because there are still old classes ranking well.

Fourthly, while the popularity of a class may play a role, David said that his “Merch By Amazon” for beginners had ranked well for over a year, “and was getting new students every day“. So, it was presumably a “popular” class, getting clicks from the results pages, new video views, new students, reviews and premium minutes. If not, why would David say, “I am losing quite a bit of money each day as a result” (of the algorithm change)?

So, if it’s not the age of a class, or the popularity of a class, what changed?

It’s hard to say. let’s dig deeper…

 

Interesting Insights…

One interesting insight is a search Sergey ran. He searched for his own name, and only one class was immediately shown at the top of the results, followed by other teacher’s classes…

 

seo1-sergey

 

When I checked, I found a block of twenty-eight of his classes near the end of the search results!

seo2-sergey

That sounds much more like a ranking penalty has been applied to those classes than a SEO algorithm tweak.

But, if Skillshare have started giving out “shadow bans”, where teachers aren’t removed from the platform, but their classes are almost impossible to find in the search results, why would one of Sergey’s classes rank at the top of the same search? Having 28/29 classes rank badly, and one rank well, makes very little sense to me. Furthermore, David, Mark and Lindsay produce great classes and I would struggle to ever see them getting a demotion.

It’s also interesting to note that, in the past, Skillshare did a “sweep” of their catalogue of classes and outright deleted many classes. There was no chance to edit a class to make it compliant. If it was gone, it was gone. I expect Skillshare was swamped with customer support tickets after taking such action, so it does make a lot of sense that, if they did another sweep, they’d institute a kind of “shadow ban” which the likes of Instagram use to suppress accounts who break their rules. When a shadowban happens, the account holder is not informed, so they’re much less likely to complain and thereby start a long dialogue exchange taking up a lot of community manager’s time. However, while this is an attractive idea, it doesn’t explain why quality teachers would see their classes demoted.

Asking Some Questions…

  • Why does a search for “Sergey Kasimov” return 373 results?
  • Why does a search for another teacher, “Alex Genadinik”, return 265 results? Neither teacher has that number of classes.
  • Why does a search for “Social Media Marketing Masterclass” return 3434 results, but “Social Media Masterclass” returns only 1593?

The answer is that, unlike most search engines, Skillshare uses the “OR” principle when searching their database. They return results for “Sergey OR Kasimov”, “Alex OR Genadinik”, and “Social OR Media OR Marketing OR Masterclass”, hence the huge number of results for that phrase! Almost all other search engines will narrow search results by using “AND” instead of “OR”, so you get fewer results with more complex search terms.

Not only do Skillshare use the “OR” functionality, they go further… they will truncate keywords to about four letters, so “Merch by Amazon” can return results by classes containing, “Amazing Fish”!

Conclusions…

Could it be that Skillshare recently made their search algorithm more lenient? Classes which used to rank well are now ranking less well because Skillshare are simply returning a bigger pool of results…? Have they sacrificed a degree of precision in the search results in favour of more results?

It certainly seems so.

What do you think…? Please leave a comment below, and share this post if you think others will enjoy it! Thanks! 🙂

Helpful links:

Update: October 2018: Skillshare didn’t revert the new search engine change, so it can only be assumed that they’re happy with the way it’s working.

Leave a Comment