There was a thread regarding the status of various issues here on Epinions. This thread was pinned to the top of the forum and locked, so that the many answers given in this thread would stay easily available for the members. As far as I can tell, the pinning and locking actually was done in early April, more than a month ago.

However, some people started posting to the locked (but not effectively locked) thread and the moderators (me mostly) have been trying to figure out what to do about it. Therefore, here is the information that we wanted to keep pinned to the top of the forum. It will take several posts, so bear with me. I will then pin AND lock this thread, so that this information can stay here in an easy to find and digest form. If you have any thoughts or points that you'd like to discuss, please feel free to bring them up in a different thread. We are not trying to censor anyone but are trying to keep useful information in one spot and easy to find.

Let me know if you have any questions about this.

--Bob

Quote: ms_pookie
Any news on the ...

- Status of the search engine,
- MPA/MPR numbers,
- Continued lack of outside hits on reviews

?

Reply by roheblius:

Here's what I have - the search engine is always improving based on what's getting found. The big switch we're trying to make is to heavily weight products with reviews more. We have some help in doing that and are trying to see if we can tailor it enough to where it can make a difference.

As for the MPR/MPA, it's a matter of finding an engineer to be able to walk through the steps backwards to find out there the issue is and to fix it so that it doesn't break again. They haven't had the time to do that yet, but it's not the type of bug that will haunt us for a long time. It just needs to be prioritized.

The outside hits is something that I don't yet have my finger on. Obviously, if you've tried googling your reviews lately, it's been harder to find Epinions stuff. When I know technically what the issue is, I'll try to pass on the information.

GG

Quote: CyndiA
Hi GG - I just wanted to double check on this. Are you saying that when you type in "foods" or click etc that the ones that have been reviewed will shift to the top now? Or that's the plan? I think that's what you mean by "weight." Reviewed items show above those not reviewed. Just wanted to check though. Tks, Cyndi

Reply by roheblius:

Yes, pretty much. The goal is to turn up the dial and make products with reviews show up higher in search. It's a little tough because there are certain searches where we simply won't have reviews and if we show reviews of the incorrect products, we will be doing shoppers a disservice. So we have to make sure the dials are correct before we can really do anything.

Quote: CyndiA
Thanks for the quick response. I was pretty sure that was what you were saying. That sounds good. I hate to hit a click and see a lot with no reviews though understand that some areas will be undercovered. I know we have sales clicks, but I'm a lot more likely to buy if I read a review first and get an idea about what others think. I'll look forward to the move up of the ones with reviews.

Reply by roheblius:

Yep, we think so too. Remember that by going to Shopping.com's search engine, it was a chance we took knowing that Shopping.com is a comparison shopping site first and foremost. That's not entirely what we are. We are a review site as well. But the first step in improving the search engine was to use their data and their technology. And now we're trying to tailor it to what we need it to be for our site. Not a fun project, but necessary based on how badly the old search performed for shoppers.

Quote: smorg
Hiya, I'm just preposterously reposting Andy's posts about the search engine from February 20, 2008. Here goes:

Reprinted from andyman (Andy Hilal):

Listen everyone, searching is an important function on the site and there's no controversy there. There's also no controversy on the subject of our current search quality. It's not good enough. I also want to ask everyone to keep perspective, though. Search is an interesting technological problem and the more I learn about it from the engineers who work on the engine, the more challenging and interesting it seems.

Let me spoil the surprise: there will always be some search queries that don't work well. Returning search results always involves some guesswork on the part of the algorithm - because for most queries there is simply an overwhelming number of lexical matches. The trick becomes not necessarily just finding, but sorting. Okay... here are 2000 matches. Which is the right one to show at the top? And sometimes those guesses are going to be wrong. I mention all of this to point out that it will always be possible to take a handful of poor examples and cry: "Search sucks!"

But we also see thousands of reviews coming in all the time, and millions of shoppers reading them and clicking out to our merchant partners. So we know that at times, the search engine does, in fact work right. It's not an either-or. You have to measure search quality as a percentage of your users' queries that are well served, and that % is never going to be 100. I'm not making excuses or denying reality, I'm just setting a cornerstone for this conversation into the ground. It has to be built from there.

So what are we working on that will actually make a measurable impact on this quality? What are the problems?

1) The search engine doesn't return products with reviews high enough. We've developed a subtle change to the search algorithm which will boost reviewed products more often. It won't be an absolute. It will help without completely overriding the lexical and behavioral relevance portions of the search algorithm. I am optimistic we will have a better user experience and more hits.

2) Even aside from the search engine, sometime site pages don't seem to know that reviews are present. A product page will say "no reviews" when in fact there are some. This is a straightforward software bug we've been chasing around. It has to be fixed in several places in order for it to work in all categories for all products. We have fixed a lot of it already. I would estimate about 65% of the problem is fixed. There are outstanding cases still, especially in media.

3) Some categories are very poorly served by search. This thread started with a travel example. And it's a perfectly valid one. But again I want us to keep perspective. We have poor search results for someone planning a trip to Brussels. This is not a reason to pack our bags, burn down the office and go home. It's room for improvement, fair and square. But we have to keep some perspective. Perhaps 10% of our queries will never work well, and that will give us hundreds of thousands of examples to talk about. It doesn't necessarily mean you throw out the baby with the bathwater.

continues by smorg...

Re-post from andyman (Andy Hilal):

So why are Epinions-only categories poorly served? To answer this, it's necessary to talk a little bit about the internal organization of the company. Most of you I think know that Epinions is a web property owned and operated by Shopping.com, which is in turn owned by eBay. When Epinions merged with DealTime and became Shopping.com, we looked for ways to stop duplicating our efforts on certain things.

For one, both companies were building product databases independently. This is difficult and costly, and it made sense to put our efforts into one and share it. Since our search experience is basically a search engine for that database, we also share that. All making sense so far, right?

And here's where Travel comes in. Shopping.com does not focus on the Travel category, or Education, or Restaurants, or some of the others we have on Epinions. SDC's business model is centered around online retail, and so is their catalog and search engine. However, we have strong communities in some of our unique categories, which we don't want to lose. So Travel remains on Epinions, but without the same support as other categories.

The search engine we borrow from Shopping.com uses many data points to do the 'guesswork' I mentioned above. One of these is 'what product do people usually click on after searching for this word.' We track what people choose on the search results page, and use this data to make our results better for the next person. This data collection takes place on Shopping.com and is deeply integrated into their website, back end, data warehouse, search indexers.... etc. It's a very complex technical system.

But since Travel doesn't show up on Shopping.com, there's no opportunity for it to acquire any user clicks. And since we use user clicks to prioritize results... Travel is behind an 8-ball. This is something we need to fix, but it's a big technical challenge. We need to take this large and complicated system for processing user clicks, and split it into a multi-headed hydra that can collect user clicks from multiple sites, and serve back different relevance scores to different sites. That is a big change and it goes way beyond Epinions. We need to get a massive amount of help from other groups in the company to make it happen. We're working on them. It's just not something that Garrett and I can snap our fingers and have tomorrow.

Still continued by smorg:

Re-post from andyman (Andy Hilal):

Epinions is a very small team inside a larger company that has its own priorities as well as obligations to eBay. We have to give and take. Sometimes we benefit greatly from having a parent company. Other times we have to compromise, and our priorities take a back seat to others in the company.

I have a lot of conversations with members that go like this:

"I have worked on databases/website so I know how difficult this would be to fix. It shouldn't be hard."

But I'd ask you all to think on this: have you ever worked inside a corporation? :) Okay, then you know that the working environment is complex and it's not always easy to rocketship a single priority right to the top. And projects take time. That's the way of it.

The other thing is that I have hundreds of these "this should be doable" conversations. And much of the time, sure, the fix really isn't that crazy hard. But with hundreds of them on our plate... the point is that any one of them could be easy but doing them all is hard. And I know from experience that even one obscure bug can ruin a member's entire day.

Still quoting Andy's update on search engine repair by smorg:

Re-posted by andyman (Andy Hilal):

We are working on all these problems, but we have to keep them in perspective, too. We are working within a larger context that doesn't always bend to our will or move quickly. And we have to prioritize - sometimes brutally so. One thing does not change: we are always doing our best to work on the most important stuff that can have the biggest impact, and constantly lobbying for more support and resources.

I understand and share all of the frustrations of the Epinions membership. I'm a member too. Let's do our best not to chicken-little ourselves on the problems we do have, focus on the biggest ones we can feasibly address, and accept that there is some level of problems that will always be with us.

More communication is a fair point, and I hope that these postings are a first step on that. We'll think through ways we can communicate to you guys what problems we're aware of and working on. But we do need some room to shift our priorities around in response to the business environment, and promising dates is always a losing game. So keep the right expectations for such communications.

Thanks for reading this far! See you on the site!

Post by millinocket:

I've had a few reviews "grayed out" (with the temporarily unavailable verbiage) - each time they reappeared with a refresh of the page. Only once did that fail and the review was back within the hour on its own.

I have no idea if that means anything - it's just a different experience from the others. I don't know which reviews they were, but I suppose a good bet is that they were movies or books.

Response by almarsezstuff (Almar):

Hello everyone,

I apologize for not responding to this right away. The reality is, there are two parts to this issue, one that isn't major and should be fixed soon, the other is probably a bigger issue and requires more research. However, both are causing the "vertical not available" thing you're seeing and I'll try to explain it as best I can. I'll break it down in two parts:

Scenario 1: It's more of a cosmetic issue. This is the less serious of the two bugs. In this scenario, the review info is greyed, but the review is still accessible through search on Epinions and via a search engine (Google, Yahoo, etc.). Since folks can access the review, it won't be missing any hits (and yes, I know about missing hits, but bear with me please) and should be credited appropriately. In terms of fixing this bug, it's hopefully just a matter of making sure the greyed out stuff is no longer grey since the reviews are indeed accessible.

Scenario 2: It's a missing product issue. This one is a bigger issue and the cause of this seems to be the product catalog. Simply put, the product wasn't available in the latest catalog update and reviews without products do not show up on the site. This in turn caused the reviews to be greyed out. Unfortunately, we can't seem to find a quick fix for this one and we're currently investigating. The worst part is, hits will be affected to reviews in this scenario if the review is not accessible. We understand that this is a major issue and we're actively trying to fix it (or at least come up with a workaround on the backend).

I hope that explains it a bit. I'm told the engineers will continue to look at the situation but I wasn't given a timeline for either fix. In the meantime, I just wanted everyone to know that we are indeed aware of the problem. I'll post more info once I get it.

-Almar

Post by Almarsezstuff (Almar):

I'll be as candid as I can in answering these questions with the knowledge I have now:

1) Is this is going to be a permanent problem (aka, are the currently dead categories dead forever);

I'm told no. In fact, when the catalog fixes go out, this should fix a lot of the problems. For the "dead forever" ones, I believe we're looking into ways to at least salvage the review and give you the ability to move it (better than the way it works now) if the product no longer exists.

2) If it's not a permanent problem, when will it be fixed?; and

The engineers are working on this right now (and trying to solve the issue of the greyed out reviews) and a bunch of other stuff including missing hits.

3) If it can be fixed, will the hits that are currently occuring but not registering going to be credited back? Will there be any IS credit and will the future, properly credited hits earn IS?

I hate to say it, but I don't think we'll be able credit back the missing hits (hopefully I'm wrong but I doubt it on this one). But if we're able to fix the missing hits, then yes, I think it's safe to assume you'll be properly credited for hits moving forward from the period it's fixed.


Please keep in mind that the answers might change depending on what they discover to be the root of the issues. As with the greyed out reviews, it turns out there was more than one cause to that but what you saw as a result was the same. Hopefully that isn't the case for everything mentioned but unfortunately it's not that straightforward (from what I'm told).

-Almar

Post by yoelc (Yoel) on June 9, 2008:

All,
We owe you all an apology for not announcing today's release sooner. We will be better about communicating upcoming releases and the impact they have on the community.

Today's release included a SPAM filter. Over the past 2 months we've been inundated with SPAM profiles promoting sex and prescription drugs sites. You would be shocked to see the numbers. This was hurting our status in Google search so we had to take drastic measures. Most new users are not setting up their profile pages, that's a secondary task. We expect that some users will object to the message which is why we asked for feedback. We will make adjustments as needed.

Again, apologies for not sharing this with the community sooner.

Thanks,
Yoel