Search indexing not currently updating

[denise]

After some investigation, mark has realized that our little website has grown up to the point where our site search setup is no longer sufficient for the amount of data y'all have generated. This is an exciting problem to have, but it also means that new posts and newly-imported posts won't be indexed into the search database until we can fix it! (And by 'we', I mean 'Mark'.)

We've got a plan that involves a new search database server (to go along with the two new webservers we ordered today), a new database schema that will make indexing, storing, and accessing search data much faster, and a method of storing and retrieving search data that makes more sense for the way usage has developed now that we have more information on what actual usage looks like. (And by 'we', I once again mean 'Mark'. There is a theme here.)

The plan is going to take a week or two to implement, though, and until then, search results will continue to be out of date. Anything that is already in the search database (as of about 10 days ago) will still be searchable, but newly-made posts and newly-imported posts won't show up until after we can fix it.

We'll let you know when the problem is fixed. Until then, continue on being awesome and making stuff: we'll catch up to you as soon as possible.

Comments

[ozqueen]

Until then, continue on being awesome and making stuff...

Same to you, DW staffers! ♥

[sharpiefan]

I feel sorry for mark, sorting issues like that single-handed. *cheers him on and offers virtual coffee and cookies*

(I'd offer The Real Thing but it'd look stalkerish. And I don't know where to send 'em anyway. *g*)

[zarhooie]

Related: back before LJ was sold to 6A, I used to send the LJ staff cookies. Trufax.

[cyrano]

I closed the ticket before I could respond, so I'll say "Awesome!" and "Thank you!" here. (:

[azurelunatic]

And thank you very much for reporting it!

There are no few organizations that have a general loathing of help tickets, because <moan> that means something's wrong and needs to be fixed </moan> -- but the general Dreamwidth support philosophy is, there are only so many of us, and sometimes we just sort of expect things to work instead of maintaining Constant Vigilance, so when someone else spots something, we are thrilled to be told about it so it can be fixed!

[zarhooie]

Yes, thanks for reporting! I was planning to respond, but I had to sleep instead. :)

[bunny_m]

Congrats! That's an awesome problem to have.


That said, a problem is still a problem that needs to be fixed. Poor Mark. ;)

And a message to *all* the DW staff and volunteers: Remember, thou art awesome!

Pretty certain you folks don't get told that nearly as much as you deserve. ^_^

[amielleon]

I had no idea you could search posts on this site. Dreamwidth just got cooler, and it didn't even change.

[sepdet]

Hopefully all the yearly renewals that just came due after last year's December influx of refugees will help cover those new server costs. :)

Thanks for keeping us updated!

[bluiis]

That is really exciting! I'm so glad I joined dreamwidth, and I love how awesome all of you are. :D

[subluxate]

Thank you, staff, for all you do, and especially thanks to Mark for all his hard work on the knotty issues like this!

[slay]

I love you for posting this because I was wondering why search hadn't been updating (I use it to stalk activity on the site in my fandoms, because it's the most beautiful tool for that ever). So thank you for keeping us informed! And super happy to hear y'all are working on a fix. ♥ You guys are amazing.

[dragondancer5150]

Just commenting to say DITTO to all that's been said before! <3 You guys are AWESOME! And by 'you', I mean 'Mark'. (JUST KIDDING!!! You're ALL awesome! <333)

[songfire]

Thank you for keeping us up-to-date! You all deserve cookies! (and someone should donate an extra pizza + free coffee to mark! :D

[zarhooie]

During Open Beta (when all of us were glued to our computers), the IRC team seriously considered ordering pizza for HQ. So what I'm saying is that it's kind of a tradition.

[songfire]

That's a great tradition! (Provided you actually follow through occasionally! XD)

I'd be happy to add a pizza/cookie/coffee bonus when I renew my paid account! :D

[azurelunatic]

Yes. (The fun starts around 18:00.)

[azurelunatic]

...dude, do you not remember the drama with the Domino's calling the wrong cellphone number? And the IRC "hey denise, pizza's here"?

[denise]

When IRC tells you that your pizza has arrived...

[denise]

I actually have hauled out the DW credit card to order pizza for Mark and his family before. The people at the pizza place were very puzzled when I tried to explain to them that I was in Baltimore, but no really, the pizza itself was going to inside their delivery radius. (When I finally got handed up to the manager, she eventually caught on to what I was doing, and thought it was very sweet.)

[sagesaria]

That is the sweetest thing I've heard today. No wonder I love having an account with you guys.

[songfire]

"The people at the pizza place were very puzzled when I tried to explain to them that I was in Baltimore, but no really, the pizza itself was going to inside their delivery radius."

*lol*

But seriously, I'll be renewing my paid account soon and would be happy to donate something extra to a pizza/cookies/coffee fund for you guys - you do so much for us!
I'll just add it to my payment! ;)

[denise]

Aw, you don't have to do that, really! Just keep renewing your account as long as you stay happy with us and that'll be good enough :)

(Or, if you really want to contribute, pick a friend -- or somebody random -- and buy time for them!)

[marahmarie]

as long as you stay happy with us

What an incredibly nice and graceful way to put it.

[msilverstar]

If you folks want any help testing it, my day job contract is up next week, but I'm still feeling the search engine love.

[marahmarie]

You know, I was wondering why when I did a search on [clucky] the other day it brought up just two titles after I'd already posted a third. I thought to myself "Is this a bug or a feature"? and stared at the result screen for the longest time. Then I went offline for two days straight. It was a nice vacation from the Internet. Then came back to see my question on this answered.

Looking forward to increased serving capacity and the "is this a bug or feature" bug getting fixed.