AI and Its Impact on Test Preparation

[Steve Linthicum]

Below is the content of an email I sent yesterday to a fellow CompTIA instructor. Thoughts?

--------------------------------------------
Another good reason to be retired:

ChatGPT Request:

Identify five standard multiple choice questions that are likely to appear in the CompTIA SY0-701 exam?

ChatGPT said:​

Sure! I can create five sample multiple-choice questions similar to what you might see on the CompTIA Security+ SY0-701 exam. These questions are designed to reflect the core domains: Attacks, Threats, Vulnerabilities; Architecture and Design; Implementation; Operations and Incident Response; Governance, Risk, and Compliance.

---

Question 1: Threats and Vulnerabilities​

A network administrator notices unusually high outbound traffic from a workstation. After investigation, they find malware that is sending sensitive data to an external server. Which type of malware is most likely responsible?
A. Ransomware
B. Trojan horse
C. Worm
D. Spyware
Answer: D. Spyware
Explanation: Spyware is designed to gather information from a system without the user’s knowledge, often sending it to external servers.

---

Question 2: Architecture and Design​

Which of the following security controls is an example of defense-in-depth?
A. Deploying a firewall
B. Implementing antivirus software
C. Using multi-factor authentication, firewalls, and intrusion detection systems together
D. Requiring strong passwords
Answer: C. Using multi-factor authentication, firewalls, and intrusion detection systems together
Explanation: Defense-in-depth involves using multiple layers of security controls to protect systems.

---

Question 3: Implementation​

An organization wants to secure wireless network communications to prevent eavesdropping. Which of the following protocols provides the strongest encryption for Wi-Fi?
A. WEP
B. WPA2 with AES
C. WPA with TKIP
D. Open Wi-Fi with VPN
Answer: B. WPA2 with AES
Explanation: WPA2 with AES is currently considered the most secure standard for Wi-Fi encryption.

---

Question 4: Operations and Incident Response​

During an incident, a security analyst wants to ensure they capture volatile data before shutting down a compromised server. Which of the following should they collect first?
A. Hard drive logs
B. RAM contents
C. Backup tapes
D. System event logs
Answer: B. RAM contents
Explanation: Volatile data like information in RAM is lost when the system is powered down, so it should be captured first during incident response.

---

Question 5: Governance, Risk, and Compliance​

Which framework is commonly used to manage information security risk and provides guidelines for creating a comprehensive cybersecurity program?
A. GDPR
B. NIST Cybersecurity Framework (CSF)
C. PCI DSS
D. HIPAA
Answer: B. NIST Cybersecurity Framework (CSF)
Explanation: The NIST CSF provides a risk-based approach for organizations to manage and improve cybersecurity.

---

If you want, I can generate 20–30 more practice questions that cover all SY0-701 domains with the same style and difficulty level as the real exam. This would be very useful for focused study.
Do you want me to do that?

 

[Gregory Childers]

CompTIA should make half the exam PBQs and interactive VM questions. Too many students are cramming for exams and using ChatGPT to pass exams with little to no understanding of the concepts.

 

[Jill West]

CompTIA should make half the exam PBQs and interactive VM questions. Too many students are cramming for exams and using ChatGPT to pass exams with little to no understanding of the concepts.

If they study too much with these kinds of questions, they won't pass anyway, even with the existing MC questions.

 

[MBA]

Yes, these types of questions are not that different from typical practice questions you find in common study guides. Which is not surprising because AI has been trained on (stolen/plagiarized) work that authors and content creators have spent ample time and effort creating. Then it simply mimics what it has seen in its training corpus when generating new content. So, I don't think this changes much in terms of what is being revealed about actual exam questions. These questions, in any case, test conceptual understanding and are not necessarily sufficient to pass without being a part of a comprehensive course. The only major impact I predict is that less and less students will resort to purchasing (overpriced) study guides and training bundles.

-Moez

 

[Gregory Childers]

Yes, these types of questions are not that different from typical practice questions you find in common study guides. Which is not surprising because AI has been trained on (stolen/plagiarized) work that authors and content creators have spent ample time and effort creating. Then it simply mimics what it has seen in its training corpus when generating new content. So, I don't think this changes much in terms of what is being revealed about actual exam questions. These questions, in any case, test conceptual understanding and are not necessarily sufficient to pass without being a part of a comprehensive course. The only major impact I predict is that less and less students will resort to purchasing (overpriced) study guides and training bundles.

-Moez

For the average person, paying for a semester-long course or a boot camp is cost-prohibitive. They are already resorting to cheap or free online videos, AI, and brain dumps.

The major impact that I predict is a huge increase in the number of people who can memorize topics, but have no practical hands-on knowledge.

And then they wonder why they have trouble finding jobs.......

 

[Steve Linthicum]

@Gregory Childers wrote: "CompTIA should make half the exam PBQs and interactive VM questions." Sounds like good advice, particularly for non-profit entities. But......

ChateGPT request:

Cost of standard multiple choice exam questions versus performance based questions

ChatGPT said:​

Got it — you’re asking about the relative cost of developing traditional multiple-choice questions (MCQs) versus performance-based questions (PBQs) for exams (like CompTIA, NBDE, etc.).
Here’s the breakdown:

---

1. Standard Multiple-Choice Questions (MCQs)​

• Development Time & Cost
  • Typically cheaper to create.
  • A subject matter expert (SME) drafts the stem, options, and key.
  • Requires psychometric review and pilot testing, but overall production is straightforward.
• Estimated Cost Range
  • Industry studies put MCQ development at $200–$1,000 per question (depending on complexity, SME rates, and review).
• Ongoing Maintenance
  • Minimal. Updates needed only if content becomes outdated.

---

2. Performance-Based Questions (PBQs)​

• Development Time & Cost
  • Much more expensive because they simulate “real-world” tasks.
  • Requires SMEs plus instructional designers, software developers, and test platform engineers.
  • Testing and validation take longer.
• Estimated Cost Range
  • Can run $2,000–$10,000+ per item, depending on:
    • Simulation complexity (drag-and-drop vs. virtual labs vs. live environment).
    • Custom coding required.
    • Scoring design (partial credit, adaptive scoring, etc.).
• Ongoing Maintenance
  • High. Software updates, platform compatibility, and content refreshes all increase lifecycle cost.

 

[Mark Anthony Germanos]

Below is the content of an email I sent yesterday to a fellow CompTIA instructor. Thoughts?

--------------------------------------------
Another good reason to be retired:

ChatGPT Request:

Identify five standard multiple choice questions that are likely to appear in the CompTIA SY0-701 exam?

ChatGPT said:​

Sure! I can create five sample multiple-choice questions similar to what you might see on the CompTIA Security+ SY0-701 exam. These questions are designed to reflect the core domains: Attacks, Threats, Vulnerabilities; Architecture and Design; Implementation; Operations and Incident Response; Governance, Risk, and Compliance.

---

Question 1: Threats and Vulnerabilities​

A network administrator notices unusually high outbound traffic from a workstation. After investigation, they find malware that is sending sensitive data to an external server. Which type of malware is most likely responsible?
A. Ransomware
B. Trojan horse
C. Worm
D. Spyware
Answer: D. Spyware
Explanation: Spyware is designed to gather information from a system without the user’s knowledge, often sending it to external servers.

---

Question 2: Architecture and Design​

Which of the following security controls is an example of defense-in-depth?
A. Deploying a firewall
B. Implementing antivirus software
C. Using multi-factor authentication, firewalls, and intrusion detection systems together
D. Requiring strong passwords
Answer: C. Using multi-factor authentication, firewalls, and intrusion detection systems together
Explanation: Defense-in-depth involves using multiple layers of security controls to protect systems.

---

Question 3: Implementation​

An organization wants to secure wireless network communications to prevent eavesdropping. Which of the following protocols provides the strongest encryption for Wi-Fi?
A. WEP
B. WPA2 with AES
C. WPA with TKIP
D. Open Wi-Fi with VPN
Answer: B. WPA2 with AES
Explanation: WPA2 with AES is currently considered the most secure standard for Wi-Fi encryption.

---

Question 4: Operations and Incident Response​

During an incident, a security analyst wants to ensure they capture volatile data before shutting down a compromised server. Which of the following should they collect first?
A. Hard drive logs
B. RAM contents
C. Backup tapes
D. System event logs
Answer: B. RAM contents
Explanation: Volatile data like information in RAM is lost when the system is powered down, so it should be captured first during incident response.

---

Question 5: Governance, Risk, and Compliance​

Which framework is commonly used to manage information security risk and provides guidelines for creating a comprehensive cybersecurity program?
A. GDPR
B. NIST Cybersecurity Framework (CSF)
C. PCI DSS
D. HIPAA
Answer: B. NIST Cybersecurity Framework (CSF)
Explanation: The NIST CSF provides a risk-based approach for organizations to manage and improve cybersecurity.

---

If you want, I can generate 20–30 more practice questions that cover all SY0-701 domains with the same style and difficulty level as the real exam. This would be very useful for focused study.
Do you want me to do that?

I tell my Security+ students they’re taking the class so they can do a job that requires cybersecurity knowledge. The Security+ certification is a prerequisite. They have to shine during the job interview and during the first 90 days at the new job, or else they will find themselves unemployed and asking “what do I do now?”