API for vitals
Large Language Model Evaluation
| Global functions | |
|---|---|
| .onLoad | Source code |
| .rlang_as_friendly_type | Source code |
| .rlang_check_is_string | Source code |
| .rlang_purrr_args_recycle | Source code |
| .rlang_purrr_index | Source code |
| .rlang_purrr_map_mold | Source code |
| .rlang_purrr_probe | Source code |
| .rlang_stop_unexpected_typeof | Source code |
| .stop_not_number | Source code |
| Task | Man page |
| accumulate | Source code |
| accumulate_right | Source code |
| accuracy | Source code |
| add_working_start_to_turns | Source code |
| add_working_times_to_turns | Source code |
| agent_chat | Source code |
| agent_progress_begin | Source code |
| agent_progress_end | Source code |
| agent_progress_update | Source code |
| agent_solver | Source code |
| agent_solvers | Man page |
| are | Man page |
| as_metadata | Source code |
| assistant_message_content | Source code |
| assistant_turn | Source code |
| assistant_turn_from_log | Source code |
| bind_token_usage | Source code |
| chat_from_log_messages | Source code |
| chat_from_model_string | Source code |
| chat_model_params | Source code |
| chat_provider_model | Source code |
| chat_provider_prefix | Source code |
| chat_set_model | Source code |
| chat_tools_list | Source code |
| check_agent_dots | Source code |
| check_arg | Source code |
| check_bool | Source code |
| check_call | Source code |
| check_character | Source code |
| check_closure | Source code |
| check_data_frame | Source code |
| check_dataset | Source code |
| check_environment | Source code |
| check_formula | Source code |
| check_function | Source code |
| check_inherits | Source code |
| check_inspect_agent_deps | Source code |
| check_log_dir | Source code |
| check_log_read_chat | Source code |
| check_logical | Source code |
| check_metrics | Source code |
| check_name | Source code |
| check_number_decimal | Source code |
| check_number_whole | Source code |
| check_sandbox | Source code |
| check_string | Source code |
| check_symbol | Source code |
| claude_code | Man page Source code |
| clean_filename_component | Source code |
| codex | Man page Source code |
| coerce_whole_numbers | Source code |
| collapse_tool_result | Source code |
| compact | Source code |
| condense_content | Source code |
| condense_sample | Source code |
| copy_dir_contents | Source code |
| copy_log_files | Source code |
| create_init_begin_event | Source code |
| create_init_end_event | Source code |
| create_model_event | Source code |
| create_sample_init_event | Source code |
| create_score_event | Source code |
| create_scorer_begin_event | Source code |
| create_scorer_end_event | Source code |
| create_solver_begin_event | Source code |
| create_solver_end_event | Source code |
| create_tool_event | Source code |
| create_tool_state_event | Source code |
| create_use_tools_begin_event | Source code |
| create_use_tools_end_event | Source code |
| detect | Source code |
| detect_answer | Man page Source code |
| detect_answer_impl | Source code |
| detect_exact | Man page Source code |
| detect_exact_impl | Source code |
| detect_includes | Man page Source code |
| detect_includes_impl | Source code |
| detect_index | Source code |
| detect_match | Man page Source code |
| detect_match_impl | Source code |
| detect_pattern | Man page Source code |
| detect_pattern_impl | Source code |
| diff_token_usage | Source code |
| discard | Source code |
| drop_nulls | Source code |
| ellmer_model_string | Source code |
| ellmer_tracks_finish_reason | Source code |
| ensure_content_list | Source code |
| errored_chat | Source code |
| eval_log | Source code |
| eval_log_filename | Source code |
| eval_log_read | Source code |
| eval_log_read_headers | Source code |
| eval_log_timestamp | Source code |
| eval_log_write | Source code |
| event_input_message | Source code |
| event_request_message | Source code |
| events_timestamp | Source code |
| every | Source code |
| extract_format_from_media_type | Source code |
| fact_default_template | Source code |
| file_url_to_path | Source code |
| generate | Man page Source code |
| generate_id | Source code |
| generate_structured | Man page Source code |
| get_api_eval_set | Source code |
| get_api_events | Source code |
| get_api_flow | Source code |
| get_api_log_dir | Source code |
| get_api_log_file | Source code |
| get_api_log_files | Source code |
| get_api_log_headers | Source code |
| get_api_logs | Source code |
| get_log_files_metadata | Source code |
| has_class | Source code |
| has_last_task | Source code |
| has_tool_calls | Source code |
| imap | Source code |
| import_inspect_log | Source code |
| import_inspect_sample | Source code |
| inject_configuration | Source code |
| input_string | Source code |
| inspect_generate_config | Source code |
| inspect_model_string | Source code |
| inspect_provider_packages | Source code |
| inspect_sandbox | Source code |
| is_content_list | Source code |
| is_old_task_id_format | Source code |
| is_positron | Source code |
| is_replaying | Source code |
| is_snapshot | Source code |
| is_testing | Source code |
| join_epochs | Source code |
| json_response | Source code |
| keep | Source code |
| key_exists | Source code |
| key_get | Source code |
| list_c | Source code |
| log_content_block | Source code |
| log_content_list | Source code |
| log_content_text | Source code |
| log_header_to_overview | Source code |
| log_input_text | Source code |
| log_model_events | Source code |
| log_sample_row | Source code |
| logged | Source code |
| map | Source code |
| map2 | Source code |
| map2_chr | Source code |
| map2_dbl | Source code |
| map2_int | Source code |
| map2_lgl | Source code |
| map_chr | Source code |
| map_dbl | Source code |
| map_if | Source code |
| map_int | Source code |
| map_lgl | Source code |
| message_content_from_turn | Source code |
| mock_chat | Source code |
| model_events_by_message_id | Source code |
| model_graded_fact | Man page Source code |
| model_graded_qa | Man page Source code |
| model_graded_qa_impl | Source code |
| move_output | Source code |
| negate | Source code |
| nonempty | Source code |
| numeric_price | Source code |
| obj_type_friendly | Source code |
| obj_type_oo | Source code |
| oxford_comma | Source code |
| parse_query_string | Source code |
| path_to_file_url | Source code |
| pmap | Source code |
| print.Task | Source code |
| process_grades | Source code |
| provider_prefix | Source code |
| py_argument_names | Source code |
| py_generate_config_names | Source code |
| qa_default_instructions | Source code |
| qa_default_template | Source code |
| qa_extract_grade | Source code |
| qa_format_prompt | Source code |
| read_log_header_json | Source code |
| reduce | Source code |
| reduce_right | Source code |
| regenerate_example_objects | Source code |
| register_agent_progress | Source code |
| rename_metric_fields | Source code |
| request_content_blocks | Source code |
| resolve_attachments | Source code |
| response_text | Source code |
| results_scores | Source code |
| scorer_chat_from_events | Source code |
| scorer_detect | Man page |
| scorer_model | Man page |
| set_id_column | Source code |
| set_reasonable_names | Source code |
| solve_with_inspect_agent | Source code |
| solver_chat | Source code |
| some | Source code |
| split_agent_args | Source code |
| split_events_at_scorer | Source code |
| split_structured_result | Source code |
| stop_input_type | Source code |
| sum_model_usage | Source code |
| task_dataset_name | Source code |
| tool_parameters_schema | Source code |
| tool_result_error | Source code |
| tool_result_from_message | Source code |
| tool_result_message | Source code |
| tool_result_value_to_content | Source code |
| translate_assistant_choices | Source code |
| translate_ellmer_content | Source code |
| translate_events_initialize | Source code |
| translate_events_scorer | Source code |
| translate_events_solver | Source code |
| translate_events_tool_use | Source code |
| translate_to_eval | Source code |
| translate_to_eval_scorers | Source code |
| translate_to_events | Source code |
| translate_to_message | Source code |
| translate_to_messages | Source code |
| translate_to_metadata_grading | Source code |
| translate_to_metrics | Source code |
| translate_to_model_usage | Source code |
| translate_to_output | Source code |
| translate_to_plan | Source code |
| translate_to_plan_steps | Source code |
| translate_to_results | Source code |
| translate_to_sample | Source code |
| translate_to_samples | Source code |
| translate_to_score | Source code |
| translate_to_stats | Source code |
| translate_to_turn_messages | Source code |
| transpose | Source code |
| turn_duration | Source code |
| turn_stop_reason | Source code |
| turn_tokens | Source code |
| turn_tool_calls | Source code |
| turn_tool_results | Source code |
| turns_from_messages | Source code |
| type_to_schema | Source code |
| vec_type_friendly | Source code |
| vitals | Man page |
| vitals-package | Man page |
| vitals_bind | Man page Source code |
| vitals_bundle | Man page Source code |
| vitals_log_dir | Man page Source code |
| vitals_log_dir_set | Man page Source code |
| vitals_log_read | Man page Source code |
| vitals_view | Man page Source code |
| vitals_view_impl | Source code |
| walk | Source code |
| warn_old_task_id_format | Source code |
| with_chat_args | Source code |
| with_response | Source code |
| write_log_dir_manifest | Source code |
| write_robots_txt | Source code |
