RAG collection manager

The RAG collection manager is an APEX region plug-in, <adm-rag-manager>, in the AI Pack. Use it to create a collection, start indexing, look at the files and chunks, and test a search. You do not have to write the configuration JSON by hand.

The plug-in shows one collection. The list of collections is an ordinary APEX Interactive Report on its own page. Select a collection in the report to open the page that holds the plug-in.

Only administrators can use it. Every method checks this on the server. A user who is not an administrator gets a no-permission error, and the region shows a short message instead of the manager.

Before you start

Install the AI Pack. See Installing the AI Pack.
Ask a developer to add the plug-in to your APEX application. The steps are in Add the plug-in to an APEX page.

Create a collection

In the collection report, select Create. The page opens with an empty form. A short text at the top says what a collection is.
Fill in the sections of the form. They follow the way a collection is built:
- Name: the name and the description.
- Which files: a folder, an annotation, or both. Fill in at least one field. With both, the collection holds the files that match either one. A line under the fields says what your entries mean.
- How text is read: Built-in reader, A model reads the file or An OCR service reads the file. The built-in reader is the default. A model or an OCR service reads only the file types that you list. Every other file keeps the built-in reader. Each new file or file version costs one call to the model or the service.
- How files are split and stored: the embedding (a Preset, or the provider, the model and the dimensions) and the Store: Oracle AI Vector Search or Qdrant. The chunk sizes and the distance metric are behind Advanced settings. The title of that box shows the values.
- How a search behaves: the switches Also run a keyword search, and merge the results (hybrid search) and Rewrite the question with a model before the search. Query rewriting adds one model call to every search. The default values of a search are behind Advanced settings.
- How questions are answered: optional. Switch on This collection writes answers to choose an answer model and a system prompt. Without an answer model, the collection returns passages.
Every model has the same settings: a provider, a model and an APEX web credential. Enter the static ID of the credential, not the key itself. The credential holds the key. The Server address, the reasoning settings for the models that can reason, and Other provider settings (JSON) are behind Advanced settings of that model. The box opens by itself when it holds a value. The keys of the other settings are described in The generic config object. A stored setting that has no field of its own shows up in the Other provider settings box, so you can see and change it.
Fix the errors that show next to a field. The server validates the configuration while you type.
Select Create collection.

If you select Create collection while something is wrong, the form lists all problems at the top. Each line names the field in words, and a line takes you to the field. The form opens an Advanced settings box that holds one of them.

Each field shows its default. A field you leave empty uses the default, as described in Creating RAG configurations.

You cannot change the embedding or the store after you create the collection. The form says so before you choose them.

Most sections have a short note with a triangle, for example More about reading files. Select it to read a short explanation. The note links to the chapter of the guide that goes deeper.

Choose a source folder

Select Browse next to Folder to pick a folder instead of typing its path:

Select the name of a folder to open it. The path bar above the list shows where you are. Select a part of it to go back up.
Select Select next to a folder, or Use with the path of the open folder, to fill the field.

A folder without subfolders cannot be opened, but you can select it. The list shows at most 500 folders. Type in Filter by name to find a folder in a larger list. If the field already has a path, the picker opens there.

Advanced (JSON)

The Advanced (JSON) tab shows the configuration that the form builds. It is the only place with key names. Use it for vector_index, which has no form field. The form keeps the content type (text) in the configuration. It has no field, because text is the only value. When you edit the JSON, select Apply to form to load your text back into the form. The server checks the text exactly as you typed it.

Change a collection

Select Edit on a collection. The form shows the stored configuration.

These fields are disabled in the form, and a note in the section How files are split and stored says so. You cannot change them after creation, because the stored vectors depend on them:

the embedding provider, model and dimensions
the vector store

An update that changes one of them fails with an invalid-state error. To use another embedding model, create a new collection.

New chunk sizes apply only to files that have no chunks yet. The files that are already indexed keep their chunks. The edit form says so above the sizes. To cut every file again with the new sizes, use Rebuild. You can also create a new collection over the same source. The course lesson Compare a larger chunk size shows the second collection.

Rebuild a collection

Rebuild is on the collection page next to Index now. It is available when no run is active. ADM asks you to confirm, and then does this:

It deletes every chunk of the collection, with its vectors. For a Qdrant collection, the points are deleted by the normal cleanup job.
It deletes the queue entries of the collection, and sets the attempts and errors of its files back to zero.
It starts indexing, as Index now does. Every file is cut into chunks with the current chunk sizes and embedded again.

The collection, its configuration and its files stay. The text of the files stays too, and other collections use it, so Rebuild does not change it. A file that has no text is read as usual. There is one exception: text from the built-in reader that no other collection has chunked. If this collection reads text with a model or OCR, it reads that text again.

Rebuild calls the embedding model for every chunk. Until indexing is done, a search finds fewer passages, or none. If a run is active, Rebuild fails with an invalid-state error and deletes nothing. The call is adm_ai_rag_admin_api.rebuild_collection. It does not start indexing. Call start_indexing after it.

Delete a collection

Delete is on the collection page, after Test search. It is available when no run is active. ADM asks you to confirm. The question names the collection. If you select Cancel, nothing is deleted.

If you confirm, ADM deletes the collection with its configuration, its file list, its chunks and its vector store. The documents in ADM are not changed. You cannot restore a deleted collection.

After the delete, the page shows a message and a Back button. The page item that holds the collection ID is set to empty. Only an administrator can delete a collection. If the call fails, the page shows the reason next to the buttons and the collection stays. The call is adm_ai_rag_admin_api.delete_collection.

Start indexing

The collection page shows a status for the collection:

Status	Meaning
`ERROR`	At least one file failed. Retry failed is available.
`PROCESSING`	Work is running now.
`PENDING`	Work is waiting.
`INCOMPLETE`	Some chunks are not embedded, or some files are not indexed yet.
`EMPTY`	The collection has no active files. Nothing matches the source, or nothing was synced yet.
`READY`	Every active file is indexed, or has no text.

A file fails when a queue job fails, or when text extraction or chunking used up its attempts. The setting AI_JOB_MAX_ATTEMPTS sets the number of attempts. While a file still has attempts left, it is not an error. The file shows the last error, and the status is Not indexed yet.

Each file has one of these statuses:

Status	Meaning
`ERROR`	A queue job failed, or text extraction or chunking used up its attempts.
`PROCESSING`	A job for the file is running.
`PENDING`	A job for the file is waiting.
`INCOMPLETE`	The file has chunks that are not embedded.
`EMPTY`	The file has no text, so it has no chunks. This does not block `READY`.
`NOT_INDEXED`	The file has no chunks yet. Text extraction or chunking has still to run.
`INDEXED`	The file has chunks and all of them are embedded.
`INACTIVE`	The file is not part of the collection any more.

The collection page shows how many files are searchable, failed, without text and not indexed yet.

Index now starts the maintenance run in a background job and returns at once. If a run is active for the collection, a second call is refused. The job runs with your ADM context.
Retry failed is available when a file failed. It resets the attempt counters, the extraction and chunking errors and the failed queue entries, and then starts a run.

The page refreshes while the collection is PROCESSING, and also while it is PENDING. Right after Index now, the status can still be PENDING. If the status stays PENDING with no run active, the page stops after 40 refreshes (about two minutes) and refreshes again after your next action or a reload.

Look at files and chunks

The collection page shows:

The header: the files, the chunks, the source (folder, annotation, or both), the embedding and the store. It also shows whether the answer model, query rewriting and hybrid search are on or off.
Backlog by stage: the files that wait at each step of indexing.
Last run and errors: the result of the last background run, and the files that failed, with the reason.
Files: filter by status or search by path. Each row shows the status, the reader of the text, the number of attempts and the error.
A warning if some files use text from another reader than the collection asks for (see below).

Select a file to see its extracted text and its chunks. Select a chunk to see the full text, the position and the length. Use Previous chunk and Next chunk to read the neighbours.

Which reader produced the text

A document version has one text. All collections share it. If another collection indexed a file first, this collection keeps that text, even when its own settings ask for OCR or a model. The Text read by column shows the reader of each file. The collection page shows a warning with the number of such files.

The text is kept because chunks point into it. To read a file with the reader of this collection, save the file as a new version. The extraction setting in the editor has a note about this.

Test a collection

Select Test search on a collection to open the debug console.

The console searches with your own access. It applies the personal access check to each chunk, with no administrator bypass: you see passages only of documents that you own or that are shared with you or your groups. An administrator who is not a member of the group that owns the source folder sees no chunk. The console does not show what another user gets.

Enter a Question. Optionally set the result limit, the score threshold, the surrounding chunks, a rewrite provider and model, or Only this document.
Run the search.

The result shows:

the query as you typed it and the query sent to the search. Not rewritten shows when rewriting is off.
the Timings of each stage.
the ranked chunks with their score. For a collection with hybrid search on, the table also shows the vector rank and the keyword rank, and a note that the score is a fused RRF value. For a vector search, the score is the similarity, and these columns and the note do not show.

Turn on the answer option to also generate an answer. The result then shows the answer, the prompts (the system prompt and the user prompt with the sources) and the sources. A test with an answer writes an entry to the debug log. See Viewing Debug Logs as Markdown.

Add the plug-in to an APEX page

Do these steps in the APEX Builder:

Make sure that the plug-in ADM_RAG_MANAGER is in your application. It is part of the ADM application export. If you build your own application, add the plug-in and upload adm-rag-manager.min.js as a file of the plug-in.
Create the list page. Use an Interactive Report on the view adm_ai_rag_collection_status_v. Link a column to the collection page and pass the collection id to the page item of that page. Add a Create button that links to the same page and clears its cache.
Create the collection page. Add a hidden page item for the collection id, for example P13110_RAG_COLLECTION_ID. Add one region of the plug-in type and select this item in the Collection ID Item attribute.
Give both pages the authorization scheme @administration-rights, and the plug-in region and the report the scheme @ai-pack.

The region reads the page item when the page renders. If the item is empty, the region shows the form for a new collection. After you save it, the region shows the new collection and sets the page item to its id. This also holds after you reload the page.

Use one region of this plug-in on a page.

Also refer to

Creating RAG configurations
adm_ai_rag_admin_api: the PL/SQL API behind the manager