favstarの中の人のActiveRecordパターンからの移行について

Kenn Ejima @kenn

@timhaines yeah forget about table_rows now, let's focus on data_length and index_length.

Tim Haines 🇺🇦 @TimHaines

@kenn Also - running a select count(*) on the dumped/reimported one took 23 secs. I gave up on the "insert into select" table after mins.

Kenn Ejima @kenn

@timhaines then maybe you did something wrong with the "insert into select" query? are you 100% sure with the query?

Tim Haines 🇺🇦 @TimHaines

@kenn I'm running another count on it now to make sure the row counts are the same. Will wait this time.

Kenn Ejima @kenn

@timhaines ok let me know when it's done... going out for lunch now. :)

Tim Haines 🇺🇦 @TimHaines

@kenn me too. :-)

Kenn Ejima @kenn

@timhaines btw, if you want to check the progress, use innotop and press R. you see how many inserts/sec is going on.

Kenn Ejima @kenn

@timhaines read records / sec, in this case

Tim Haines 🇺🇦 @TimHaines

@kenn Wow - super slow. http://t.co/JTsy4Y4j

拡大

Scott Mayo @toxaq

@timhaines you dealing in billions of lines again?

Tim Haines 🇺🇦 @TimHaines

@toxaq about 300m for this table I think. 20GB before the remove, 7GB after.

Kenn Ejima @kenn

@timhaines I don't know how slow it looks to you. :) why did you want to insert-select in the first place? ALTER TABLE does that for you.

Tim Haines 🇺🇦 @TimHaines

@kenn I thought it would give me fresh clean table the same way dump did, but allowed me to keep the original too.

Kenn Ejima @kenn

@timhaines ah, i see. but the original will be stale immediately as the new table gets new records? for just-in-case I'd just keep the dump

Tim Haines 🇺🇦 @TimHaines

@kenn I'm mucking about on a paused slave trying to figure out how I can dump and restore to single file per table with 24 hours.

Kenn Ejima @kenn

@timhaines tablespace will never shrink. you need another fresh slave with file_per_table config

Tim Haines 🇺🇦 @TimHaines

@kenn Yeah, that's what I'm setting up. Sorry, described it wrong.

Kenn Ejima @kenn

@timhaines i see. btw you gave up on Riak? :)

Tim Haines 🇺🇦 @TimHaines

@kenn Yep. Riak write performance was terrible with LevelDB.

Kenn Ejima @kenn

@timhaines nice tip. I'm leaning toward polyglot persistence - innodb+mongodb+redis ftw

Tim Haines 🇺🇦 @TimHaines

@kenn I use redis quite a bit too, for some timelines etc that fit in ram.

Kenn Ejima @kenn

@timhaines yeah, redis is the coolest thing in recent years. I use mysql+redis in every project.

🖐🏽🎈phark 🖐🏽🎈 @pharkmillups

@timhaines @kenn 1.2 release has bloom filter support and better compaction which should show better performance for some use cases.

Kenn Ejima @kenn

@pharkmillups for bloom filter how do you compare it with redis? redis has super-easy bitset support

Tim Haines 🇺🇦 @TimHaines

@pharkmillups @kenn @dizzyd tested writing new objects with the bloom filter branch too - it didn't help apparently.