(10:21:42) kosh: I trust zope to save things, however I setup the data so that it is more efficient to save (10:21:46) kosh: so I worry about what really matters (10:21:59) peterb1: like relational data? (10:22:13) kosh: no (10:22:22) tiredbones left the room (quit: Read error: 60 (Operation timed out)). (10:22:48) kosh: for instance don't store a whole bunch of basic attributes (strings, lists, dicts etc) inside an object you will have to load a lot if you don't need all the items inside it all the time (10:23:14) xbeanx: good tip (10:23:14) kosh: the zodb will load up all second class objects when the parent is opened but all objects that inherit from persistent are only loaded when they are accessed directly (10:23:29) kosh: stuff like that REALLY cuts down on disk io to load objects (10:23:44) kosh: another thing is only write a var back to itself if it has changed (10:23:53) jbb666 left the room (quit: ). (10:23:55) kosh: that speeds up saving and dramatically slows db growth (10:24:22) xbeanx: Are most zope developers still developing for Zope 2? (10:24:25) sm: good point.. kind of shame to break object encapsulation though, if something belongs to an object and you store it elsewhere (10:24:40) kosh: ie if you just keep hitting the save button you should not be doing io in general (10:24:43) sm: you have to keep them in sync, etc. (10:24:50) kosh: sm: no reason to break encapsulation, just put it in a subobject (10:25:00) sm: aha (10:25:06) sm: interesting! (10:25:10) kosh: sm: for instance you put metadata for a picture directly in your object but the file data as a subobject (10:25:26) kosh: sm: that way you can get the metadata for display purposes easily but only load the file when it is actually requests (10:25:44) sm: this is something to think about for zwiki page revisions (10:25:48) peterb1: like extfile :) (10:26:22) CrippsFX [n=Cripps@leonid2.housing.mun.ca] entered the room. (10:26:23) xbeanx: kosh: interesting, I'm actually working with image objects right now.. (10:26:26) ***sm thinks.. guess the way it is now is also fine (10:26:33) kosh: other simple thing like doing write lots of changes to a dict (10:26:41) kosh: if you make a change to a simple object the entire object is saved again (10:27:03) CrippsFX left the room (quit: Client Quit). (10:27:07) kosh: so if you write several hundred changes to a dict one transaction at a time you are looking at probably 100M or more of memory usage (10:27:16) kosh: however if you used an OOBTree you are probably about 2M (10:27:18) CrippsFX [n=Cripps@leonid2.housing.mun.ca] entered the room. (10:27:34) kosh: it is because an OOBTree can write changes while a dict works like that penny a day doubled thing (10:28:09) sm: I wonder if I could halve zwiki memory usage by saving the prerendered text as a subobject instead of an attribute (10:28:57) kosh: probably help a fair bit (10:29:03) sm: OOBTree.. where would you use that ? as a container? (10:29:09) kosh: however it is probably even better to make them both subobjects (10:29:20) kosh: that way to load one you never have to load the other (10:29:34) kosh: anyplace you would use a dict you can use an OOBTree
summary: zwiki page objects could be split into more zodb objects, so that eg touching a page object to check it's attributes doesn't require loading up the prerendered data from zodb.