Commit Graph

612 Commits

Author SHA1 Message Date
Par Winzell 918405ec02 While investigating problems, add much paranoid profiling. Also let's do an explicit keyset copy during enumeration rather than the fancy concurrent hashmaps and iterable mappings. Finally, let's fully synchronize the other maps we're using. 2009-01-24 18:53:42 +00:00
Par Winzell db33c89b66 Get rid of some awful debug bits. 2009-01-23 19:51:12 +00:00
Par Winzell fc52b277a3 The CacheAdapter interface has changed:
* No more CacheBins, operations happen directly on the adapter.
 * Each cache write must supply a CacheCategory value that identifies its type. Current values are RECORD, KEYSET and RESULT. We will probably need a MISC or USER or something along those lines, too.

The EHCache adapter has been entirely rewritten:
 * Each CacheCategory maps to one EHCache, which means there are only a very few EHCaches, and we know what they are. We expect these to be declared in ehcache.xml rather than programmatically generated as before.
 * Elements inside each EHCache are indexed by (cacheId, elementKey) tuples.

The main purpose of this refactor is to sort all Records into a single EHCache, which we can then make really, really large, and then let the LRU mechanism sort out what should be in memory and what shouldn't. The previous implementation would make no ram allocation distinction between busy records and ones rarely read.
2009-01-23 16:12:11 +00:00
Par Winzell 07d848a3a0 Oops, we need to replicate what Depot does elsewhere to construct a database-unique identifier for the index. 2009-01-23 15:17:51 +00:00
Par Winzell c654606a2d Add a convenience migration for dropping indexes. 2009-01-23 15:05:08 +00:00
Par Winzell 0569f8f8ff The buildIndex() magic expects an array of colums, not a list. 2009-01-22 16:43:21 +00:00
Par Winzell 7dbc58d426 Function and Literal expressions can be non-deterministic but in most cases they are safe, and giving them proper toString()'s will help caching quite a bit. 2009-01-21 21:39:48 +00:00
Par Winzell 017d9242ac Better log message. 2009-01-12 15:13:34 +00:00
Par Winzell 062a8efcac Let the EHCache adapter be a little more configurable. 2009-01-10 01:47:11 +00:00
Par Winzell a5de1295f9 Let the EHCache adapter be a little more configurable. 2009-01-10 01:46:50 +00:00
Michael Bayne a8b3e24e87 Allow caching to be skipped for single record loads, ensure that we don't cache
DepotMigrationHistoryRecord.
2009-01-09 21:40:30 +00:00
Michael Bayne c46535cffd Nixed unused imports, variables. 2009-01-08 22:15:46 +00:00
Michael Bayne 53358e74c9 Nixed unused import. 2009-01-08 22:08:35 +00:00
Par Winzell 60c681ff74 Introduce a new cache strategy that does collection queries in two stages, but does not cache the keyset. This is a safe strategy, and we use it in places where an out-of-date result set is dangerous or annoying. 2009-01-08 18:22:40 +00:00
Michael Bayne 87b547743c Moved some implemention detail clauses into impl.claus. I'd like to move
SelectClause in there as well, but peskily two things on MSOY are using it to
do subselects. I'd prefer there was some less "reaching into the internals" way
to do subselects.
2009-01-08 02:31:28 +00:00
Michael Bayne a9c992bb44 Great big revampage: no more strings anywhere. It's all ColumnExps all the way
down (well, all the way to the public API anyway).

Revamped index creation while I was in there because that was one of the big
string users. Now you just put @Index on the field you want indexed, and if you
want a multi-column index you do things with a magical static method just like
we do for complex (function) indices.

@UniqueConstraint may still go away since it's basically exactly the same thing
as @Index(unique=true), so it's kind of pointless to support both.
2009-01-08 02:10:54 +00:00
Michael Bayne a66f15e355 Minor edits. 2009-01-07 19:36:43 +00:00
Par Winzell f5fc65f1e1 More cache configuration options for findAll(), especially a new CONTENTS strategy which will cache the entire result set of the supplied query, without the restrictions of the KEYS approach. This is very lovely and perhaps a little scary. 2009-01-07 19:32:24 +00:00
Par Winzell 269a3ffb92 Let's make this an insertion-ordered set so that all logically identical queries get assigned table abbreviations in the same order. This is mostly cosmetic, but has no significant cost. 2009-01-01 17:10:34 +00:00
Par Winzell 68aac7fd11 Progress notification during migration soothes the soul. 2009-01-01 15:47:42 +00:00
Par Winzell c41534c7e6 We need these two fellows for the previous patch to make sense. 2008-12-31 00:59:28 +00:00
Par Winzell 0ccd8bdb82 Extend our Index annotation to accept a 'complex' attribute. If an index ixFoo is marked complex for a given record, Depot will call the static method ixFooDefinition on that record class and expect a List<Tuple<SQLExpression, OrderBy.Order>> return value.
The purpose of all this is to create function (or expression, rather) indexes, which PostgreSQL does quite happily.
2008-12-31 00:25:29 +00:00
Michael Bayne e3aed39330 Document our thrown exceptions. 2008-12-16 00:00:52 +00:00
Par Winzell 44fd01cc2d Switch to nulls here, too, making migration work right for HSQLDB. 2008-12-10 23:02:34 +00:00
Par Winzell b3cb1d6214 Cut and paste error. 2008-12-10 21:09:56 +00:00
Michael Bayne 50d31cfcc5 Unused import nixage. 2008-12-10 20:05:34 +00:00
Par Winzell f0f08f9516 More HSQL support:
- HSQL doesn't have full text search, so we just OR up a bunch of LIKE conditions to catch any and all matches. This is not efficient, but it'll do just fine for things like unit tests and development work.
 - HSQL doesn't have & and |, it calls them BITAND and BITOR.
 - HSQL doesn't have MySQL's unix_timestamp() or a way to get an 'epoch' from a date like PostgreSQL, so we count the number of seconds since 1970-01-01. I have not investigated to make sure this is exact, but it should at least be internally consistent.
 - HSQL has a very neat list of data types that (no surprise) map very well to Java types.
2008-12-09 23:26:11 +00:00
Par Winzell 96dfa3ca04 Break up the function that generates ColumnDefinitions for our fields and pass around a mutable instance of that object so as to give subclasses more freedom to do whatever weird stuff they need to to generate SQL for all the incompatible dialects out there. This is not elegant, and I feel this code along with ColumnDefinition and some of DatabaseLiaison, should pperhaps find a new home in the SQLBuilder structure. 2008-12-09 23:20:14 +00:00
Par Winzell b9bb84d3af Cope with the new ColumnDefinition. 2008-12-09 23:18:17 +00:00
Par Winzell 7eaaea15e7 Instead of enumerating columns for which values will be auto-generated and supplying magical DEFAULT arguments for them, simply don't include those columns in the insert. 2008-12-09 23:00:50 +00:00
Par Winzell cc07817238 Remove an ancient TODO: Get rid of SELECT FOR UPDATE and instead locklessly claim the next available primary key value. 2008-12-09 22:56:55 +00:00
Par Winzell 817857d20e HSQL doesn't think 'false' is a proper condition in a DELETE statement, and I suppose it has a point. This should always work. 2008-12-09 20:28:20 +00:00
Michael Bayne 76268a7fb1 This should have been deleted. 2008-12-01 18:17:14 +00:00
Michael Bayne 14a85d0de6 Differentiate between explicit queries and decomposed queries. 2008-11-25 22:02:15 +00:00
Michael Bayne 8ab95ec10e Unconcretification of QueryClause. I concreted it a while back when it did
something and then those needs went away and now it's just a marker that
differentiates arbitrary SQLExpression instances from those that represent
clauses of a query.
2008-11-25 20:53:33 +00:00
Michael Bayne 8fec576663 Moved things that are not part of the Depot public API into an impl package.
Also moved WhereClause into depot.clause since it really wanted to be there.
2008-11-25 20:44:22 +00:00
Michael Bayne 980be19918 Unused import pruning and other cleaning. 2008-11-25 20:14:09 +00:00
Michael Bayne 0b439978fb Stubbed out Depot support for HSQLDB. This also doesn't work yet but will some
day be made to work and enable more sophisticated unit testing of Depot and
systems that use Depot.
2008-11-25 20:13:43 +00:00
Michael Bayne bc44c4e58e A cache adapter for testing. 2008-11-25 00:57:22 +00:00
Michael Bayne 9e2dbeb174 Finished collection caching implementation. Our databases will shortly breath a
great sigh of relief.
2008-11-25 00:57:04 +00:00
Michael Bayne ae34697c8d KeySet is going to get stuffed into the cache, so we're making it more memory
efficient.

Also switched from Collection to Iterable in some internal spots where we don't
need anything other than an iterator. I wanted to push this all the way out to
DepotRepository but ran into some snags. I'm leaving in this change in case I
feel like desnaggling the external bits later.
2008-11-24 22:05:04 +00:00
Michael Bayne ae414006bc These don't *really* override anything so we can't mark them as such as JDK 1.5
will do le freak out.
2008-11-21 06:30:32 +00:00
Michael Bayne f6339978bf Missed a spot. 2008-11-21 06:11:57 +00:00
Michael Bayne 475200849c Extract query and record cache tracking into parent. 2008-11-21 03:19:44 +00:00
Michael Bayne e347f4ac17 More toString()ery. Changed KeyUtil to DepotUtil so I could jam some more
useful stuff in it.
2008-11-21 03:15:47 +00:00
Michael Bayne e98ad09588 If loadAll() can return its results entirely from the cache before invoke(),
have it do so. Down to the last couple of steps for collection caching.
2008-11-21 02:50:16 +00:00
Michael Bayne 22c086f64e Extracted findAllKeys() query into a helper class, made it possible for Query
to request a read-write connection.
2008-11-21 02:30:08 +00:00
Michael Bayne 3cf711e083 More toString()s. 2008-11-21 02:04:07 +00:00
Michael Bayne 9880ff326c A whole bunch of revampery on the way to collection caching, including some
basic stats reporting and query and cache logging.

PersistenceContext no longer sticks its nose so deeply into the business of
Query and Modifier. It just passes itself along and allows them to do any cache
activity they need during their normal execution. The one situation where we
keep our nose in their business is to allow a Query to return a result from the
cache before it is even invoked so that we can avoid requesting a database
connection if we won't ever need one.
2008-11-21 01:49:28 +00:00
Michael Bayne 5f26eb28c9 Fixed sed error. 2008-11-20 00:16:57 +00:00