Skip to content

doc update for obsproc manual reflecting S25A prep #102

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
monodera merged 1 commit into from
Sep 13, 2024
Merged

doc update for obsproc manual reflecting S25A prep #102

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 34 additions & 13 deletions docs/examples/obsproc.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,9 @@ data_dir = "<path to the data directory on the uploader>"

## Working with filter names, proposal categories, and target types

### Insert to the `filter_name`, `proposal_category`, and `target_type` tables
### Insert to the `filter_name`, `proposal_category`, `pfs_arm`, and `target_type` tables

`filter_name`, `proposal_category`, and `target_type` tables are expected to be very static and not frequently updated.
`filter_name`, `proposal_category`, `pfs_arm`, and `target_type` tables are expected to be very static and not frequently updated.
Note that you are most likely to skip this step as these tables are already populated in the database.

The contents of CSV files to be inserted for these tables are as follows:
Expand Down Expand Up @@ -73,6 +73,14 @@ proposal_category_id,proposal_category_name,proposal_category_description
4,uh,University of Hawaii proposal
```

```csv title="pfs_arm.csv"
name,description
b,"blue"
r,"red"
n,"near-infrared"
m,"medium resolution red"
```

```csv title="target_types.csv"
target_type_id,target_type_name,target_type_description
1,SCIENCE,the fiber is intended to be on a science target
Expand All @@ -92,6 +100,7 @@ You can insert these data into the database using the following commands:
```console
$ pfs-targetdb-cli insert filter_names.csv -c db_config.toml --table filter_name --commit
$ pfs-targetdb-cli insert proposal_categories.csv -c db_config.toml --table proposal_category --commit
$ pfs-targetdb-cli insert pfs_arm.csv -c db_config.toml --table pfs_arm --commit
$ pfs-targetdb-cli insert target_types.csv -c db_config.toml --table target_type --commit
```

Expand Down Expand Up @@ -160,15 +169,15 @@ Suppose you have an Excel file named `pfs_allocation_summary.xlsx` with 2 sheets
**`Proposals` Sheet**:

| proposal_id | input_catalog_name | input_catalog_description | group_id | pi_first_name | pi_last_name | pi_middle_name | proposal_category_name | upload_id | n_obj | fiberhour_total | fiberhour_lr | fiberhour_mr | rot_total | rot_lr | rot_mr |
|-------------|--------------------|---------------------------|----------|---------------|--------------|----------------|------------------------|------------------|-------|-----------------|--------------|--------------|-----------|--------|--------|
| ----------- | ------------------ | ------------------------- | -------- | ------------- | ------------ | -------------- | ---------------------- | ---------------- | ----- | --------------- | ------------ | ------------ | --------- | ------ | ------ |
| S99A-QT001 | pfs_example_1 | Example target list 1 | o99101 | Eiichi | Shibusawa | | openuse | d6e94eae259faf4e | 1572 | 379.5 | 379.5 | | 5.2 | 5.2 | |
| S99A-QT002 | pfs_example_2 | Example target list 2 | o99102 | Umeko | Tsuda | | openuse | 5f695375c60f34c7 | 9712 | 17504 | 17504 | | 15.83 | 15.83 | |
| S99A-QT003 | pfs_example_3 | Example target list 3 | o99103 | Shibasaburo | Kitasato | | openuse | ba59115da8084653 | 2047 | 395.25 | 395.25 | | 12.7 | 12.7 | |

**`Allocation` Sheet**:

| proposal_id | grade | rank | allocated_rot_total | allocated_rot_lr | allocated_rot_mr | allocated_time_total | allocated_time_lr | allocated_time_mr | n_ppc | allocation_rate_lr | allocation_rate_mr | completion_rate_lr | completion_rate_mr |
|-------------|-------|------|---------------------|------------------|------------------|----------------------|-------------------|-------------------|-------|--------------------|--------------------|--------------------|--------------------|
| ----------- | ----- | ---- | ------------------- | ---------------- | ---------------- | -------------------- | ----------------- | ----------------- | ----- | ------------------ | ------------------ | ------------------ | ------------------ |
| S99A-QT001 | A | 9 | 2.8 | | 2.8 | 284.25 | 0 | 284.25 | 9 | 0.749011858 | | 0.723 | |
| S99A-QT002 | B | 6.5 | 6.5 | 6.5 | | 8140.5 | 8140.5 | 0 | 21 | 0.465065128 | | 0.279 | |
| S99A-QT003 | B | 6 | 9.6 | 9.6 | | 350.25 | 350.25 | 0 | 31 | 0.886148008 | | 0.684 | |
Expand All @@ -182,20 +191,19 @@ $ pfs-targetdb-cli parse-alloc pfs_allocation_summary.xlsx
The command will generate the following CSV files in the current directory.

```csv title="proposal.csv"
proposal_id,group_id,pi_first_name,pi_last_name,pi_middle_name,rank,grade,allocated_time_total,allocated_time_lr,allocated_time_mr,proposal_category_name
S99A-QT001,o99101,Eiichi,Shibusawa,,9.0,A,284.25,0.0,284.25,openuse
S99A-QT002,o99102,Umeko,Tsuda,,6.5,B,8140.5,8140.5,0.0,openuse
S99A-QT003,o99103,Shibasaburo,Kitasato,,6.0,B,350.25,350.25,0.0,openuse
proposal_id,group_id,pi_first_name,pi_last_name,pi_middle_name,rank,grade,allocated_time_total,allocated_time_lr,allocated_time_mr,proposal_category_name,is_too
S99A-QT001,o99101,Eiichi,Shibusawa,,9.0,A,284.25,0.0,284.25,openuse,false
S99A-QT002,o99102,Umeko,Tsuda,,6.5,B,8140.5,8140.5,0.0,openuse,false
S99A-QT003,o99103,Shibasaburo,Kitasato,,6.0,B,350.25,350.25,0.0,openuse,true
```

```csv title="input_catalogs.csv"
input_catalog_name,input_catalog_description,upload_id,proposal_id
pfs_example_1,Example target list 1,d6e94eae259faf4e,S99A-QT001
pfs_example_2,Example target list 2,5f695375c60f34c7,S99A-QT002
pfs_example_3,Example target list 3,ba59115da8084653,S99A-QT003
input_catalog_name,input_catalog_description,upload_id,proposal_id,is_classical
pfs_example_1,Example target list 1,d6e94eae259faf4e,S99A-QT001,false
pfs_example_2,Example target list 2,5f695375c60f34c7,S99A-QT002,false
pfs_example_3,Example target list 3,ba59115da8084653,S99A-QT003,true
```


### Insert to the `proposal` and `input_catalog` tables for `target` data

You can insert them into the database using the following commands:
Expand All @@ -213,6 +221,19 @@ You need to transfer the target lists from the uploader to the local storage.
$ pfs-targetdb-cli transfer-targets input_catalogs.csv -c db_config.toml
```

At the end of the command, a summary will be shown as follows.

```
upload_id status n_transfer
d6e94eae259faf4e success 1
5f695375c60f34c7 WARNING 2
ba59115da8084653 FAILED 0
```

You should look at the `WARNING` and `FAILED` entries to complete the transfer.
In the cases above, `WARNING` is shown because there are more than one directories with `upload_id=5f695375c60f34c7 ` transfered (`n_transfer=2`).
The `FAILED` is shown because it failed to find an appropriate directory.

This command will transfer the target lists from the uploader to the local storage.
For example, you will see the following directories in the current directory:

Expand Down