SecAI+ AI Red Teaming Exercise Basics

1. In AI red teaming, what is 'adversarial example generation'?

Creating deliberately crafted inputs that fool AI models into incorrect outputs

Generating random test data

Writing documentation for security teams

Scheduling security training sessions

2. What is a 'red team exercise' timeline typically focused on?

Simulating real-world attack scenarios with defined scope, duration, and objectives

Permanently shutting down AI systems

Training employees on coffee break etiquette

Scheduling annual vacation periods

3. Which defense mechanism helps protect against prompt injection attacks?

Input sanitization and strict prompt templates

Disabling all user inputs

Running AI systems offline only

Removing all security logging

4. What is the purpose of 'robustness testing' in AI red teaming?

Ensuring models maintain performance under adversarial or unexpected conditions

Testing physical hardware durability

Verifying user interface aesthetics

Checking database backup procedures

5. In red teaming, which technique involves testing how an AI model handles out-of-distribution inputs?

Fuzzing

Model extraction

Gradient analysis

Cache poisoning

6. What is a 'backdoor' in the context of AI model security?

A hidden mechanism allowing unauthorized access or control of the model

The main entrance to a data center

A backup storage location

A user logout feature

7. Which of the following best describes 'responsible disclosure' in red teaming?

Publicly announcing vulnerabilities immediately

Keeping vulnerabilities secret indefinitely

Reporting findings to the organization with time for remediation before public release

Selling vulnerability information to the highest bidder

8. In red teaming exercises, what does 'guardrail testing' evaluate?

Whether safety mechanisms and restrictions in AI systems hold under attack

Physical safety of data centers

Employee behavior compliance

Network bandwidth limitations

9. What is 'input validation' as a defense against red teaming attacks?

Checking and filtering user inputs to prevent malicious instructions

Ignoring all user feedback

Removing all user interface elements

Storing unencrypted user data

10. Which stakeholder group should receive red teaming findings first?

Marketing department

Security and development teams responsible for remediation

Company executives only

External competitors

11. What is the primary goal of AI red teaming in cybersecurity?

To attack production systems without authorization

To identify vulnerabilities and weaknesses in AI systems before adversaries exploit them

To replace traditional penetration testing entirely

To monitor user activity on social media

12. What does 'threat modeling' accomplish in AI red teaming?

Identifies potential attack vectors and security risks specific to AI systems

Removes all security vulnerabilities automatically

Prevents all cyberattacks permanently

Monitors employee email activity

13. Which of the following is a key metric used to evaluate red teaming success?

Number of vulnerabilities discovered

Time to detect attacks

Impact severity and exploitability

All of the above

14. What is the primary difference between red teaming and traditional penetration testing?

Red teaming focuses on AI-specific vulnerabilities and behavioral risks

Penetration testing is only for physical security

Red teaming requires no authorization

There is no meaningful difference

15. In red teaming, an 'evasion attack' involves which of the following?

Avoiding detection by security systems through adversarial examples

Stealing model weights directly

Shutting down the AI application

Modifying firewall rules

16. What is 'model poisoning' in the context of AI security?

Corrupting training data to degrade model performance or introduce backdoors

Slowing down model inference intentionally

Encrypting model parameters

Removing all model documentation

17. Which phase of red teaming focuses on gathering information about the target AI system?

Exploitation

Reporting

Reconnaissance

Remediation

18. What is a 'prompt injection' attack?

Injecting malware into network packets

Inserting malicious instructions into user prompts to manipulate AI behavior

Physically inserting devices into servers

Injecting false data into training datasets

19. In AI red teaming, what does 'jailbreaking' refer to?

Bypassing safety guidelines and restrictions in AI models

Physically breaking AI hardware

Removing encryption from data

Deleting model training logs

20. Which of the following is a common adversarial attack technique used in red teaming?

Prompt injection

Model poisoning

Evasion attacks