favstarの中の人のActiveRecordパターンからの移行について

@timhaines @pharkmillups @kenn https://t.co/sUm3tLMV

2012-05-18 05:52:29

@dizzyd @pharkmillups @kenn There's no significant throughput improvement there right? Am I misinterpreting?

2012-05-18 05:54:52

@timhaines @pharkmillups @kenn Personally, I think that 400 ops/sec after inserting 27G of data is a nice speedup vs. 150 ops/sec @ 1G.

2012-05-18 05:56:29

@dizzyd @pharkmillups @kenn Yes, that's true. Graph doesn't show 150 ops/s, so I didn't see it?

2012-05-18 05:57:33

@timhaines @pharkmillups @kenn Let me be succinct. Bloom filter does make a dramatic improvement. My measurements were wrong on Friday...

2012-05-18 05:58:34

@timhaines @pharkmillups @kenn but someone was more interested in tweeting about it than giving me a chance to nail down root cause.

2012-05-18 05:59:09

@dizzyd sorry if using your name wasn't appropriate. It's still comparatively poor write perf compared to mysql/cass on same HW tho right?

2012-05-18 06:01:38

@dizzyd https://t.co/vDKZByKw

2012-05-18 06:04:55

@kenn The results are in. Both tables have 253623565 rows. Dump/import count: 24 sec. "Insert t_new select" count: 1 hour 31 min 58.86 sec

2012-05-18 06:06:59

@timhaines wow. take diff of "show create table"

2012-05-18 06:08:34

@kenn they're identical apart from the table name.

2012-05-18 06:09:27

@timhaines super weird. even spooky.

2012-05-18 06:10:03

@kenn I'm thinking the "insert into select" probably does some weird (de-)optimization as part of the query.

2012-05-18 06:11:02

@timhaines that's unlikely, but at this point, I have to agree on what should be the culprit.

2012-05-18 06:11:48

@timhaines what does explain say? same result for both count query?

2012-05-18 06:13:06

@kenn Ahh - the fast table would have been in the buffer pool when I did the count most likely. 2nd one wasn't.

2012-05-18 06:13:39

@timhaines ah. :) then if you run it now, it should only take 23 sec?

2012-05-18 06:14:09

@kenn actually, that's not right. I just recounted the first one, and it finished in 51 sec again - almost as fast.

2012-05-18 06:14:29

@timhaines buffer pool scenario makes sense. problem solved? :)

2012-05-18 06:15:14

@timhaines all I know is that 1.2 will be a lot faster than 1.1 for that specific test. Use the tool that works best for you.

2012-05-18 06:18:10

@timhaines but didn't you say the size was different, too? that's the weirdest part to me.

2012-05-18 06:21:25

@kenn yes, size is different, and count time is very different.

2012-05-18 06:25:00

@timhaines same explain result?

2012-05-18 06:30:07

@kenn there's no problem (for me) btw. I'm dumping / reimporting them all anyway, to get to file per table.

2012-05-18 06:31:34