Contact Information

L3
ModelContextProtocolFilesystemDesktop Template

Extract contact details from various file formats on desktop and perform reasoning analysis on the collected relationship data.

Created by Lingjun Chen
2025-08-14
Data ExtractionCross Referencing

Model Ranking

Click on the dots to view the trajectory of each task run
Model
Run Results
Pass@4
Pass^4
Avg Time
Avg Turns
Input Tokens
Output Tokens
Total Tokens
Grok
grok-4
4
/4
183.5s
5.5
31,439
9,069
40,508
OpenAI
gpt-5-low
2
/4
200.8s
7.8
33,411
11,505
44,917
Claude
claude-sonnet-4
1
/4
149.6s
12.0
75,325
2,815
78,140
Gemini
gemini-2-5-pro
1
/4
59.4s
12.0
49,870
3,684
53,554
OpenAI
gpt-5-medium
1
/4
197.1s
6.3
55,100
9,560
64,660
OpenAI
o3
1
/4
193.0s
37.0
171,688
13,080
184,767
Claude
claude-opus-4-1
0
/1
--
161.2s
10.0
54,985
2,735
57,720
Claude
claude-sonnet-4-high
0
/4
72.2s
14.8
101,091
3,516
104,607
Claude
claude-sonnet-4-low
0
/4
76.4s
14.0
90,807
3,359
94,165
DeepSeek
deepseek-chat
0
/4
236.3s
25.8
142,469
2,728
145,197
Gemini
gemini-2-5-flash
0
/4
34.1s
6.5
19,718
4,321
24,039
Z.ai
glm-4-5
0
/4
79.5s
16.0
65,956
2,943
68,899
OpenAI
gpt-4-1
0
/4
36.2s
11.8
35,944
1,983
37,927
OpenAI
gpt-4-1-mini
0
/4
56.7s
27.3
93,072
1,970
95,042
OpenAI
gpt-4-1-nano
0
/4
20.4s
10.8
32,009
856
32,864
OpenAI
gpt-5-high
0
/4
507.6s
7.8
104,894
14,449
119,342
OpenAI
gpt-5-mini-high
0
/4
73.3s
8.3
118,773
8,098
126,871
OpenAI
gpt-5-mini-low
0
/4
44.7s
9.0
55,376
2,922
58,298
OpenAI
gpt-5-mini-medium
0
/4
64.0s
15.0
146,695
5,077
151,772
OpenAI
gpt-5-nano-high
0
/4
104.7s
14.8
75,798
20,298
96,095
OpenAI
gpt-5-nano-low
0
/4
102.0s
36.0
203,414
14,348
217,762
OpenAI
gpt-5-nano-medium
0
/4
72.2s
10.8
34,211
11,952
46,163
OpenAI
gpt-oss-120b
0
/4
11.8s
6.0
12,620
401
13,022
Grok
grok-code-fast-1
0
/4
54.7s
14.3
114,288
1,606
120,789
MoonshotAI
kimi-k2-0711
0
/4
141.3s
16.3
78,252
2,064
80,315
MoonshotAI
kimi-k2-0905
0
/4
163.4s
19.3
89,951
2,071
92,022
OpenAI
o4-mini
0
/4
461.4s
22.5
74,190
14,618
88,808
Qwen
qwen-3-coder-plus
0
/4
62.5s
31.0
160,353
2,969
163,321
Qwen
qwen-3-max
0
/4
36.8s
12.3
65,881
1,213
67,094

Task State

Task Initial State Files
Download ZIP package to view the complete file structure
desktop_template/ ├── Archives/ │ ├── backup_contacts.csv │ └── tax_documents_2022.csv ├── Desktop/ │ └── contacts.csv ├── Documents/ │ ├── Personal/ │ │ └── tax_info_2023.csv │ ├── Projects/ │ │ └── budget_tracker.csv │ ├── Work/ │ │ ├── client_list.csv │ │ └── timesheet.csv │ ├── budget.csv │ └── important_dates.csv ├── Downloads/ │ ├── expenses.csv │ ├── fitness_log.csv │ └── price_comparisons.csv ├── Temp/ │ └── test_data.csv ├── book_list.txt ├── bookmark_export.txt ├── calculations.txt ├── correspondence_2023.txt ├── draft_letter.txt ├── emergency_contacts.txt ├── example.txt └── experiment_results.txt

Instruction

Please use FileSystem tools to finish the following task:

Task Description

Your task is to compile all contact information from all the files into a single CSV table. You need to extract all people's contact information and organize it systematically.

Task Objectives

  1. Scan all files in the directory
  2. Extract contact information for all individuals and organizations found
  3. Create a CSV file named contact_info.csv in the main directory
  4. Structure the CSV with the following columns:
    • First column: Name (required)
    • Second column: Email (required)
    • Third column: Phone (required)
    • Additional columns: Any other contact information types found
  5. Consolidate information by merging the same types of information into single columns
  6. Leave cells blank if specific information is not available for a person/organization

Expected Output

  • File name: contact_info.csv
  • Format: CSV with headers and data rows

Reasoning Task

After creating the contact_info.csv file, analyze the data to answer: What is Charlie Davis's job/profession?

Hint: focus on the contact information in contact_info.csv.

Write your answer in a file named answer.txt in the main directory.

Important Notes

  • Do not modify any existing files
  • Only create the two new files: contact_info.csv and answer.txt


Verify

*.py
Python
#!/usr/bin/env python3
"""
Verification script for Contact Information Compilation Task
"""

import sys
from pathlib import Path
import csv
import os
import re

def get_test_directory() -> Path:
    """Get the test directory from FILESYSTEM_TEST_DIR env var."""
    test_root = os.environ.get("FILESYSTEM_TEST_DIR")
    if not test_root:
        raise ValueError("FILESYSTEM_TEST_DIR environment variable is required")
    return Path(test_root)

def verify_contact_info_csv_exists(test_dir: Path) -> bool:
    """Verify that the contact_info.csv file exists in the main directory."""
    contact_file = test_dir / "contact_info.csv"
    
    if not contact_file.exists():
        print("❌ File 'contact_info.csv' not found in main directory")
        return False
    
    print("✅ contact_info.csv file found")
    return True

def verify_answer_txt_exists(test_dir: Path) -> bool:
    """Verify that the answer.txt file exists in the main directory."""
    answer_file = test_dir / "answer.txt"
    
    if not answer_file.exists():
        print("❌ File 'answer.txt' not found in main directory")
        return False
    
    print("✅ answer.txt file found")
    return True

def verify_csv_structure(test_dir: Path) -> bool:
    """Verify that the CSV file has the correct structure."""
    contact_file = test_dir / "contact_info.csv"
    
    try:
        with open(contact_file, 'r', encoding='utf-8') as f:
            reader = csv.reader(f)
            rows = list(reader)
            
        if len(rows) < 2:  # Need at least header + 1 data row
            print("❌ CSV file has insufficient rows")
            return False
        
        headers = rows[0]
        if not headers:
            print("❌ CSV file has no headers")
            return False
        
        # Check that Name is the first column
        if headers[0].lower() != 'name':
            print("❌ First column is not 'Name'")
            return False
        
        # Check that Email and Phone are present (order may vary)
        header_lower = [h.lower() for h in headers]
        if 'email' not in header_lower:
            print("❌ 'Email' column not found")
            return False
        
        if 'phone' not in header_lower:
            print("❌ 'Phone' column not found")
            return False
        
        print("✅ CSV structure is correct")
        return True
        
    except Exception as e:
        print(f"❌ Error reading CSV file: {e}")
        return False

def verify_csv_content_accuracy(test_dir: Path) -> bool:
    """Verify that the CSV content contains all required data, regardless of row order or extra entries."""
    contact_file = test_dir / "contact_info.csv"
    
    try:
        with open(contact_file, 'r', encoding='utf-8') as f:
            reader = csv.DictReader(f)
            rows = list(reader)
        
        # Expected data from answer.csv (hardcoded as required)
        expected_data = [
            {"Name": "John Smith", "Email": "john@email.com", "Phone": "555-0101", "Status": "", "Industry": ""},
            {"Name": "Jane Doe", "Email": "jane@email.com", "Phone": "555-0102", "Status": "", "Industry": ""},
            {"Name": "Bob Johnson", "Email": "bob@email.com", "Phone": "555-0103", "Status": "", "Industry": ""},
            {"Name": "Alice Brown", "Email": "alice@email.com", "Phone": "555-0201", "Status": "Inactive", "Industry": ""},
            {"Name": "Charlie Davis", "Email": "charlie@email.com", "Phone": "555-0202", "Status": "Active", "Industry": ""},
            {"Name": "David Wilson", "Email": "david@email.com", "Phone": "555-0203", "Status": "Inactive", "Industry": ""},
            {"Name": "Acme Corp", "Email": "acme@corp.com", "Phone": "", "Status": "", "Industry": "Technology"},
            {"Name": "Global Inc", "Email": "global@inc.com", "Phone": "", "Status": "", "Industry": "Finance"},
            {"Name": "Local Business", "Email": "local@biz.com", "Phone": "", "Status": "", "Industry": "Retail"},
            {"Name": "Spouse", "Email": "", "Phone": "+1-555-0124", "Status": "", "Industry": ""},
            {"Name": "Parent", "Email": "", "Phone": "+1-555-0125", "Status": "", "Industry": ""},
            {"Name": "Sibling", "Email": "", "Phone": "+1-555-0126", "Status": "", "Industry": ""},
            {"Name": "Primary Doctor", "Email": "", "Phone": "+1-555-0201", "Status": "", "Industry": ""},
            {"Name": "Dentist", "Email": "", "Phone": "+1-555-0202", "Status": "", "Industry": ""},
            {"Name": "Pharmacy", "Email": "", "Phone": "+1-555-0203", "Status": "", "Industry": ""}
        ]
        
        # Convert expected data to a dictionary for easier lookup
        # We'll use Name as the key since it should be unique
        expected_dict = {}
        for entry in expected_data:
            expected_dict[entry["Name"]] = entry
        
        # Check each row for accuracy, regardless of order
        # Allow extra entries and mixed content
        found_entries = set()
        extra_entries = []
        
        for i, row in enumerate(rows):
            row_name = row.get('Name', '')
            if not row_name:
                # Skip rows without names (they're not valid entries)
                continue
            
            if row_name in expected_dict:
                # This is one of our expected entries
                if row_name in found_entries:
                    print(f"❌ Duplicate name found: '{row_name}'")
                    return False
                
                found_entries.add(row_name)
                expected = expected_dict[row_name]
                
                # Check all columns for this entry
                for key, expected_value in expected.items():
                    if key in row:
                        actual_value = row[key] if row[key] else ""
                        if actual_value != expected_value:
                            print(f"❌ Entry '{row_name}', column '{key}': expected '{expected_value}', got '{actual_value}'")
                            return False
                    else:
                        print(f"❌ Entry '{row_name}' missing column '{key}'")
                        return False
            else:
                # This is an extra entry - record it for informational purposes
                extra_entries.append(row_name)
        
        # Verify all expected entries were found
        if len(found_entries) != len(expected_data):
            missing = set(expected_dict.keys()) - found_entries
            print(f"❌ Missing entries: {missing}")
            return False
        
        # Report extra entries if any
        if extra_entries:
            print(f"ℹ️  Found {len(extra_entries)} extra entries: {extra_entries}")
        
        print(f"✅ CSV content accuracy verified: found all {len(expected_data)} required entries (plus {len(extra_entries)} extra entries)")
        return True
        
    except Exception as e:
        print(f"❌ Error verifying CSV content: {e}")
        return False

def verify_csv_data_completeness(test_dir: Path) -> bool:
    """Verify that all required data is present and no entries are missing."""
    contact_file = test_dir / "contact_info.csv"
    
    try:
        with open(contact_file, 'r', encoding='utf-8') as f:
            reader = csv.DictReader(f)
            rows = list(reader)
        
        # Check that all expected names are present
        expected_names = [
            "John Smith", "Jane Doe", "Bob Johnson", "Alice Brown", 
            "Charlie Davis", "David Wilson", "Acme Corp", "Global Inc", 
            "Local Business", "Spouse", "Parent", "Sibling", 
            "Primary Doctor", "Dentist", "Pharmacy"
        ]
        
        actual_names = [row.get('Name', '') for row in rows if row.get('Name')]
        
        missing_names = set(expected_names) - set(actual_names)
        if missing_names:
            print(f"❌ Missing names: {missing_names}")
            return False
        
        extra_names = set(actual_names) - set(expected_names)
        if extra_names:
            print(f"⚠️  Extra names found: {extra_names}")
            # This is a warning, not an error
        
        print("✅ CSV data completeness verified")
        return True
        
    except Exception as e:
        print(f"❌ Error checking data completeness: {e}")
        return False

def verify_answer_content(test_dir: Path) -> bool:
    """Verify that the answer.txt contains the correct answer about Charlie Davis."""
    answer_file = test_dir / "answer.txt"
    
    try:
        content = answer_file.read_text().strip().lower()
        
        # The answer should contain "dentist" (as per answer.txt)
        if "dentist" in content:
            print("✅ Answer about Charlie Davis's job is correct")
            return True
        else:
            print(f"❌ Answer does not contain 'dentist'. Found: '{content}'")
            return False
        
    except Exception as e:
        print(f"❌ Error reading answer.txt: {e}")
        return False

def verify_file_locations(test_dir: Path) -> bool:
    """Verify that files are in the correct locations."""
    contact_file = test_dir / "contact_info.csv"
    answer_file = test_dir / "answer.txt"
    
    # Check that files are in the main directory, not in subdirectories
    if contact_file.parent != test_dir:
        print(f"❌ contact_info.csv is not in main directory: {contact_file}")
        return False
    
    if answer_file.parent != test_dir:
        print(f"❌ answer.txt is not in main directory: {answer_file}")
        return False
    
    print("✅ Files are in correct locations")
    return True

def main():
    """Main verification function."""
    test_dir = get_test_directory()
    print("🔍 Verifying Contact Information Compilation Task...")
    
    # Define verification steps
    verification_steps = [
        ("Contact Info CSV Exists", verify_contact_info_csv_exists),
        ("Answer TXT Exists", verify_answer_txt_exists),
        ("Files in Correct Locations", verify_file_locations),
        ("CSV Structure", verify_csv_structure),
        ("CSV Content Accuracy (Flexible)", verify_csv_content_accuracy),
        ("CSV Data Completeness", verify_csv_data_completeness),
        ("Answer Content", verify_answer_content),
    ]
    
    # Run all verification steps
    all_passed = True
    for step_name, verify_func in verification_steps:
        print(f"\n--- {step_name} ---")
        if not verify_func(test_dir):
            all_passed = False
    
    # Final result
    print("\n" + "="*50)
    if all_passed:
        print("✅ Contact Information Compilation Task completed successfully!")
        print("🎉 Task verification: PASS")
        sys.exit(0)
    else:
        print("❌ Task verification: FAIL")
        sys.exit(1)

if __name__ == "__main__":
    main()