Andrew Kane
ca10cbaa7d
Revert "Remove offsethash"
...
This reverts commit 1cbd204f52
2024-02-20 16:07:32 -08:00
Heikki Linnakangas
1cbd204f52
Remove offsethash
...
The original motivation was to eliminate the superfluous HnswPtrAccess
call from AddToVisited. The caller has to call HnswPtrAccess() anyway,
so it makes sense to pass the HnswElement rather than HnswElementPtr
to AddToVisited(). But then I realized that we can use the
pointer-variant even with shared memory, because the visited-hash is
backend-private, and the addresses where the elements are mapped to in
shared memory are stable within the backend.
2024-02-19 14:12:26 +02:00
Andrew Kane
5ba62fca84
Fixed crash with shared_preload_libraries - fixes #460
2024-02-14 17:13:30 -08:00
Andrew Kane
ded649891b
DRY HNSW tuple alloc size [skip ci]
2024-01-28 22:58:38 -08:00
Heikki Linnakangas
121f53b8ff
Remove unused heapRel arguments ( #443 )
2024-01-26 12:45:10 -08:00
Andrew Kane
f7eda7bd20
DRY neighbor array size [skip ci]
2024-01-24 17:54:33 -08:00
Heikki Linnakangas
c8be3a369b
Include generic_xlog.h directly in the .c files where it's needed
...
There are no references to anything that's in generic_xlog.h in the
header files.
2024-01-23 13:04:03 +02:00
Heikki Linnakangas
e5d1a6bdbb
Include reloptions.h directly in the .c files where it's needed
...
There are no references to anything that's in reloptions.h in the
header files. They need to include genam.h instead, which defines
IndexScanDesc.
2024-01-23 13:02:24 +02:00
Heikki Linnakangas
f31d708c2b
Add direct include to pairingheap.h in headers
...
ivfflat.h and hnsw.h have references to pairingheap_node, so they need
to include lib/pairingheap.h. It happened to work, because
lib/pairingheap.h was being included indirectly through
nodes/execnodes.h, but let's be explicit.
Remove the include from hnswbuild.c, because there are no calls to
pairingheap functions in that file. Instead, add the includes to
hnswutils.c and ivfscan.c, which do have such calls. They are not
strictly necessary again because of the indirect include from hnsw.h
and ivfflat.h, but let's be explicit while we're messing with this.
2024-01-23 12:53:22 +02:00
Andrew Kane
a1e526ef82
Dropped support for Postgres 11
2024-01-22 23:52:54 -08:00
Andrew Kane
2d0f162bd7
Added support for in-memory parallel index builds for HNSW
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2024-01-22 23:19:10 -08:00
Heikki Linnakangas
4c6928bd3c
Remove HnswSpool
...
It was just used to pass heap/index relations to
HnswParallelScanAndInsert. I think it was copied from nbtsort.c, which
is more complicated. I don't think we need a struct like this.
(That said, I actually think that we should have a state object that
would hold fields like 'heap', 'index', 'procinfo', 'collation'
etc. Passing that object around would simplify the signatures of many
functions. But that's a different story).
2024-01-22 23:11:25 -08:00
Heikki Linnakangas
6fd05dd6f6
Remove unused 'scantuplesortstates' field
2024-01-22 23:08:20 -08:00
Andrew Kane
44b90be452
Made variable name consistent across functions [skip ci]
2024-01-22 19:02:33 -08:00
Andrew Kane
c7d60346d8
Improved macro [skip ci]
2024-01-13 20:02:41 -08:00
Andrew Kane
597bfdc76b
Added HnswGetNeighbors macro
2024-01-13 20:00:34 -08:00
Andrew Kane
cbf3eb4fa5
Improved HNSW build and insert code
2024-01-13 10:07:42 -08:00
Andrew Kane
1881b857f9
Simplified code
2024-01-09 18:53:31 -08:00
Andrew Kane
9e680884bd
Moved indtuples to HnswGraph
2024-01-07 22:23:49 -08:00
Andrew Kane
19a0e1b341
Moved graph to separate struct
2024-01-07 20:15:30 -08:00
Andrew Kane
cb4c770df2
Switched to slist for elements to reduce allocations and remove limit
2024-01-07 18:26:19 -08:00
Andrew Kane
8ee37b60a0
Improved memory estimate for HNSW index builds
2024-01-03 13:47:50 -05:00
Andrew Kane
9b73b3d1a6
Reduced memory and allocations for heap TIDs - closes #385
2024-01-03 13:41:34 -05:00
Andrew Kane
e8c3bf0cef
Improved memory tracking for HNSW index builds - #384
2023-12-22 13:35:43 -05:00
Andrew Kane
dcbe0b6f0d
Reduced memory usage for HNSW index builds - #384
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2023-12-22 12:41:47 -05:00
Andrew Kane
6738fa0bd7
Added HNSW_MEMORY flag to show memory usage - #384 [skip ci]
2023-12-20 16:49:16 -05:00
Andrew Kane
e88a425c9b
Reduced WAL generation for HNSW index builds - thanks @hlinnaka
2023-12-19 20:37:32 -05:00
Andrew Kane
921427ee03
Replace dynahash hash table in HNSW with simplehash for speed - #378
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2023-12-17 11:24:13 -05:00
Andrew Kane
dfee5d4045
Added support for on-disk parallel index builds for HNSW
2023-11-11 19:29:45 -08:00
Andrew Kane
94f7304ccd
Keep vector for now to be overly cautious about packing [skip ci]
2023-11-09 18:43:55 -08:00
Andrew Kane
d078db3d25
Switched HnswElementTuple to generic data and zero full section
2023-11-09 18:28:25 -08:00
Andrew Kane
3cf6f62900
Switched to datum for HnswElement
2023-11-09 17:35:39 -08:00
Andrew Kane
84e073888c
Removed vector-specific code from HNSW_ELEMENT_TUPLE_SIZE [skip ci]
2023-11-09 16:57:01 -08:00
Andrew Kane
3f3463bde5
Improved memory calculation for HNSW and removed vector-specific code
2023-11-09 16:21:26 -08:00
Andrew Kane
a713e2acaa
Improved performance of HNSW index builds - closes #292
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2023-10-05 13:21:26 -07:00
Andrew Kane
6e1312ddbe
DRY max size
2023-10-04 21:43:34 -07:00
Andrew Kane
bca50a03fa
Use consistent variable name
2023-09-12 19:24:31 -07:00
Andrew Kane
b164833933
Removed pinning for HNSW index scan
2023-09-11 12:12:28 -07:00
Andrew Kane
d05d6ee83d
Get m from metapage
2023-09-03 01:35:21 -07:00
Jonathan S. Katz
e50a79108f
Set default HNSW "ef_construction" to 64 ( #230 )
2023-08-26 16:05:08 -07:00
Andrew Kane
e406b7f5ea
Added comments [skip ci]
2023-08-23 21:03:07 -07:00
Andrew Kane
6d88a9e1d2
Updated HNSW_PAGE_ID [skip ci]
2023-08-21 22:59:53 -07:00
Andrew Kane
1e851c12c0
Updated comment [skip ci]
2023-08-21 22:55:24 -07:00
Andrew Kane
8ed3cc5f0b
Improved macro [skip ci]
2023-08-21 22:52:58 -07:00
Andrew Kane
bace0891bd
Updated comments [skip ci]
2023-08-21 22:51:24 -07:00
Andrew Kane
0e1de45463
Improved locking code [skip ci]
2023-08-21 15:42:59 -07:00
Andrew Kane
3f3b3ca8e3
Made function static [skip ci]
2023-08-21 03:27:42 -07:00
Andrew Kane
02f4e0ec8b
Revert "Added version to reduce stale reads and writes and prepare for optimistic locking"
...
This reverts commit ef1209eaf4
2023-08-21 02:47:27 -07:00
Andrew Kane
ef1209eaf4
Added version to reduce stale reads and writes and prepare for optimistic locking
2023-08-20 17:08:20 -07:00
Andrew Kane
687263ccd4
DRY HNSW vacuum code
2023-08-20 14:52:31 -07:00
