Heikki Linnakangas
121f53b8ff
Remove unused heapRel arguments ( #443 )
2024-01-26 12:45:10 -08:00
Heikki Linnakangas
571697fee7
Remove unnecessary UpdateProgress() wrappers ( #433 )
...
Now that we require PostgreSQL v12, we can use
pgstat_progress_update_param directly.
2024-01-25 00:07:57 -08:00
Heikki Linnakangas
c8be3a369b
Include generic_xlog.h directly in the .c files where it's needed
...
There are no references to anything that's in generic_xlog.h in the
header files.
2024-01-23 13:04:03 +02:00
Heikki Linnakangas
f31d708c2b
Add direct include to pairingheap.h in headers
...
ivfflat.h and hnsw.h have references to pairingheap_node, so they need
to include lib/pairingheap.h. It happened to work, because
lib/pairingheap.h was being included indirectly through
nodes/execnodes.h, but let's be explicit.
Remove the include from hnswbuild.c, because there are no calls to
pairingheap functions in that file. Instead, add the includes to
hnswutils.c and ivfscan.c, which do have such calls. They are not
strictly necessary again because of the indirect include from hnsw.h
and ivfflat.h, but let's be explicit while we're messing with this.
2024-01-23 12:53:22 +02:00
Heikki Linnakangas
a1b1c99ff7
Remove unused #include
...
pg_list.h has not been used in hnswbuild.c since commit cb4c770df2
2024-01-23 12:53:13 +02:00
Andrew Kane
a1e526ef82
Dropped support for Postgres 11
2024-01-22 23:52:54 -08:00
Andrew Kane
8ffb3718a4
Leave more space for other shared memory
2024-01-22 23:31:55 -08:00
Andrew Kane
2d0f162bd7
Added support for in-memory parallel index builds for HNSW
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2024-01-22 23:19:10 -08:00
Heikki Linnakangas
4c6928bd3c
Remove HnswSpool
...
It was just used to pass heap/index relations to
HnswParallelScanAndInsert. I think it was copied from nbtsort.c, which
is more complicated. I don't think we need a struct like this.
(That said, I actually think that we should have a state object that
would hold fields like 'heap', 'index', 'procinfo', 'collation'
etc. Passing that object around would simplify the signatures of many
functions. But that's a different story).
2024-01-22 23:11:25 -08:00
Heikki Linnakangas
6fd05dd6f6
Remove unused 'scantuplesortstates' field
2024-01-22 23:08:20 -08:00
Andrew Kane
597bfdc76b
Added HnswGetNeighbors macro
2024-01-13 20:00:34 -08:00
Andrew Kane
cbf3eb4fa5
Improved HNSW build and insert code
2024-01-13 10:07:42 -08:00
Andrew Kane
65d060ac86
Reverted FlushPages pattern for parallel builds
2024-01-08 10:45:31 -08:00
Andrew Kane
62ee33bb92
Improved locking code
2024-01-08 09:05:12 -08:00
Andrew Kane
520e274dde
Improved locking code
2024-01-07 22:34:41 -08:00
Andrew Kane
9e680884bd
Moved indtuples to HnswGraph
2024-01-07 22:23:49 -08:00
Andrew Kane
19a0e1b341
Moved graph to separate struct
2024-01-07 20:15:30 -08:00
Andrew Kane
c7fe1571ee
Improved code
2024-01-07 18:30:51 -08:00
Andrew Kane
cb4c770df2
Switched to slist for elements to reduce allocations and remove limit
2024-01-07 18:26:19 -08:00
Andrew Kane
85fdecd79b
Moved FlushPages before HnswEndParallel
2024-01-07 17:50:46 -08:00
Andrew Kane
6132428914
Improved number of parallel workers for HNSW index builds - closes #397
2024-01-05 19:46:08 -08:00
Andrew Kane
81d13bd40f
Improved code [skip ci]
2024-01-03 13:53:23 -05:00
Andrew Kane
8ee37b60a0
Improved memory estimate for HNSW index builds
2024-01-03 13:47:50 -05:00
Andrew Kane
cae630784b
Improved BuildCallback [skip ci]
2023-12-30 20:55:29 -05:00
Andrew Kane
d87bcd2deb
Added comments [skip ci]
2023-12-30 18:29:01 -05:00
Andrew Kane
736576220a
Improved BuildCallback
2023-12-30 18:24:03 -05:00
Andrew Kane
1e422cd62b
Improved readability [skip ci]
2023-12-22 16:39:13 -05:00
Andrew Kane
569c69580a
Improved InsertTuple code - #384
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2023-12-22 15:08:28 -05:00
Andrew Kane
59509c3a17
Added extra 5% to memory estimate
2023-12-22 14:04:05 -05:00
Andrew Kane
61738846af
Updated comment [skip ci]
2023-12-22 14:03:33 -05:00
Andrew Kane
e8c3bf0cef
Improved memory tracking for HNSW index builds - #384
2023-12-22 13:35:43 -05:00
Andrew Kane
50d1aed3d8
Improved memory usage logging [skip ci]
2023-12-22 13:09:11 -05:00
Andrew Kane
66e14d2434
Updated indentation [skip ci]
2023-12-22 12:59:50 -05:00
Andrew Kane
42cd4c6833
Fixed call to GenerationContextCreate for Postgres < 15
2023-12-22 12:49:07 -05:00
Andrew Kane
dcbe0b6f0d
Reduced memory usage for HNSW index builds - #384
...
Co-authored-by: Heikki Linnakangas <heikki.linnakangas@iki.fi >
2023-12-22 12:41:47 -05:00
Andrew Kane
f61d4087b5
Slightly improved memory estimation [skip ci]
2023-12-21 10:31:36 -05:00
Andrew Kane
57554e5b46
Added todo [skip ci]
2023-12-20 17:52:31 -05:00
Andrew Kane
6738fa0bd7
Added HNSW_MEMORY flag to show memory usage - #384 [skip ci]
2023-12-20 16:49:16 -05:00
Andrew Kane
43e0b3d9d4
Mark buffer contents as dirty when not logging
2023-12-20 16:16:25 -05:00
Andrew Kane
2bff7ccaa2
Improved memory estimation - #384 [skip ci]
2023-12-20 11:10:27 -05:00
Andrew Kane
e88a425c9b
Reduced WAL generation for HNSW index builds - thanks @hlinnaka
2023-12-19 20:37:32 -05:00
Andrew Kane
a59aa02dd9
Only show message if flushed due to memory [skip ci]
2023-12-04 15:29:23 -08:00
Andrew Kane
2fef497b7e
Fixed check
2023-12-04 15:22:12 -08:00
Andrew Kane
a156f6c7ae
Fixed invalid memory alloc request size error with HNSW - fixes #43
2023-12-04 15:00:32 -08:00
Andrew Kane
dfee5d4045
Added support for on-disk parallel index builds for HNSW
2023-11-11 19:29:45 -08:00
Andrew Kane
c5e8c46b80
Switched from VECTOR_SIZE to VARSIZE_ANY [skip ci]
2023-11-09 19:41:38 -08:00
Andrew Kane
fbb904ae2f
Use pointer for VARSIZE_ANY
2023-11-09 17:50:28 -08:00
Andrew Kane
3cf6f62900
Switched to datum for HnswElement
2023-11-09 17:35:39 -08:00
Andrew Kane
84e073888c
Removed vector-specific code from HNSW_ELEMENT_TUPLE_SIZE [skip ci]
2023-11-09 16:57:01 -08:00
Andrew Kane
3f3463bde5
Improved memory calculation for HNSW and removed vector-specific code
2023-11-09 16:21:26 -08:00
