Skip to content

I usually get a lot of questions about quiz availability and time restrictions. In particular, I am asked to clarify the quiz availability and timing options. I’m writing this blog post to provide some clarification on this.

The availability settings are used by Brightspace to determine when to make the link to the quiz available for the students to click on to start the exam. The timing options are used to determine how much time the student will have to take the exam. Availability and timing work independently of each other. If the availability date/time is set for a two hour exam window, then the student will be able to start the exam anytime during that two hour window and the timing will determine how long they have to take the exam. Brightspace IS NOT going to subtract the time the student started the exam and then limit the testing period to the remainder of the time. A student will have the entire time to take the exam regardless of when they started it.

Example of availability dates for an exam
Example of availability dates for an exam

The way in which the availability and timing options work may be more of a concern for faculty who are teaching a remote class and their students are expected to take their exams on a given day at a given time. For example, you have a course that is being taught remotely. Your class meets 8:00 am until 9:15 am on Tuesdays and Thursdays. Your students are expected to take their exams during the regular class meeting time. You can restrict the time students have to start the exam. You will have to decide how long students would have to start the exam. Let’s assume you are okay with the students starting their exam during the first 15 minutes of the class meeting time. The start time for this example would be 8 am on the testing day. The end time would be set to 8:15 am on the testing day. This means the students would have this 15 minute window to start their exam.

Example of an enforced time limit for an exam
Example of an enforced time limit for an exam

Your timing options would be set to give the students 75 minutes to complete the exam once they start it. A student who starts the exam at 8:10 am would have until 9:25 am to complete it.

A student who tries to start the exam after 8:15 am will NOT be able to start the exam because the link will no longer be available. Likewise, a student who has an internet connectivity issue while taking the exam will not be able to get back into the exam after 8:15 am (to pick up where they left off) because the link will no longer be available. In those cases, you would have to use the “Add Users to Special Access” option to grant the student access to the exam. The Add Users to Special Access option would allow you to change the length of time the student has to take the test and/or to open the exam back up for a student who had connectivity issues.

Continuing with my example, you have a student who notifies you at 8:30 am that they are late and want to be able to take the exam. You would edit the exam, click on the Add Users to Special Access button (on the Restrictions tab), and change the timing to allow the student the remaining 45 minutes to complete the exam. You would also have to change the end date/time for that particular student's special access in order for the link to the exam to be available to that student.

NOTE: The type of access would NOT be changed when allowing the student to take the exam. The “Type of Access” would still be “Allow selected users special access to this quiz”. More information about Special Access for quizzes can be found in this blog post.

Example of special access for an exam
Example of special access for an exam

Finally, I think it is a good idea to explain the availability and timing options that you decide to set for your exams to your students. This could eliminate confusion about the testing options and what to expect when taking the exam.

Want more information?

About Quizzes
Birghtspace Tip #207: Quizzes - Special Access
Grant Special Access to Users for a Quiz (video)
Best Practices: The Quiz Tool
Quizzes FAQ

View all the Brightspace training recaps
Instructors Quick Start Tutorial
Continuous Delivery release notes
Brightspace Known Issues
Request a sandbox course
Sign-up for Brightspace training sessions
You can find Brightspace help at D2L's website.
Join the Brightspace Community.
Try these Brightspace How-To documents.
Visit our Brightspace FAQs for additional Brightspace information
or schedule a one-on-one session, email, or
call Janice Florent: (504) 520-7418.

Note: Are you doing something innovative in Brightspace or perhaps you've discovered a handy tip? Share how you are using Brightspace in your teaching and learning in The Orange Room.

Did you know Special Access is an option available in the Assignments tool that allows the instructor to grant learners special accommodations? Special Access permissions allow instructors to set different availability dates and times for specific learners. For example, you could extend the deadline for learners who require remedial help or who are submitting work beyond the original scope of the assignment. You can also add Special Access after an assignment’s end date has passed for learners who have a legitimate excuse for missing the deadline or for learners you want to submit additional material, such as planning notes or a bibliography.

Special Access using the New Assignment Creation Experience
Add Special Access in the New Assignment Creation Experience

Special Access using the classic experience
Add Special Access in the classic experience

manage special access options

  1. Select Allow users with special access to submit outside the normal availability dates for this folder if you want to give certain students different submission options from the rest of the class.
  2. Select Allow only users with special access to see this folder if only certain students are allowed to see this folder.
  3. Click Add Users to Special Access to select students who will get the special access.

Want more information?

Allow Special Access for Assignments
Set Release Conditions and Special Access in Assignments
Brightspace Tip #142: Simplify Assignment Collection

View all the Brightspace training recaps
Instructors Quick Start Tutorial
Continuous Delivery release notes
Brightspace Known Issues
Request a sandbox course
Sign-up for Brightspace training sessions
You can find Brightspace help at D2L's website.
Join the Brightspace Community.
Try these Brightspace How-To documents.
Visit our Brightspace FAQs for additional Brightspace information
or schedule a one-on-one session, email, or
call Janice Florent: (504) 520-7418.

Note: Are you doing something innovative in Brightspace or perhaps you've discovered a handy tip? Share how you are using Brightspace in your teaching and learning in The Orange Room.

Did you know Special Access is an option available in the Quizzes tool that allows the instructor to grant learners special accommodations? Special Access allows the instructor to assign specific learners a different set of quiz availability dates, late submission properties, and/or time limits. The instructor can change Start, End, and Due dates; the "Recommended Time Limit" and "Enforced Time Limit" timing settings; and the number of attempts allowed for a quiz to provide allowances for different needs of learners.

special access options in quizzes
Special Access options in Quizzes

  1. Select Allow selected users special access to this quiz if you want to give certain students different submission options from the rest of the class.
  2. Select Allow only users with special access to see this quiz if only certain students are allowed to see this quiz.
  3. Click Add Users to Special Access to select students who will get the special access.

NOTE: Instructors can exempt individual learners from the requirement to use the Respondus LockDown Browser when taking a quiz. Follow these instructions to add Special Access to quizzes that require Respondus LockDown Browser (RLDB).

Want more information?

Special Access in Quizzes
Grant Special Access to Users for a Quiz (video)
Add Special Access to quizzes that require Respondus LockDown Browser
Brightspace Tip #214: Quizzes – Set Accommodations by Learner for All Quizzes

View all the Brightspace training recaps
Instructors Quick Start Tutorial
Continuous Delivery release notes
Brightspace Known Issues
Request a sandbox course
Sign-up for Brightspace training sessions
You can find Brightspace help at D2L's website.
Join the Brightspace Community.
Try these Brightspace How-To documents.
Visit our Brightspace FAQs for additional Brightspace information
or schedule a one-on-one session, email, or
call Janice Florent: (504) 520-7418.

Note: Are you doing something innovative in Brightspace or perhaps you've discovered a handy tip? Share how you are using Brightspace in your teaching and learning in The Orange Room.

Updated: 2/28/2021

Respondus LockDown Browser (RLDB) has a new remote proctoring option that works with Zoom, Microsoft Teams, Google Meet and other web based video systems. The new 'Instructor Live Proctoring' option uses LockDown Browser to prevent cheating on the computer itself, while the instructor watches students via video conferencing. The Instructor Live Proctoring method is recommended for small class sizes (20-25 students max) where instructors can effectively observe and track students during the exam.

Zoom Class Meeting

When Respondus LockDown Browser is used with Instructor Live Proctoring, the instructor and students meet in the video conferencing application at a specified time. When ready, the instructor provides students a start code for the online exam – students start LockDown Browser from there. The instructor can then watch students via the video conference system for the duration of the exam. Note that Instructor Live Proctoring DOES NOT record the student video, nor does it alert the instructor to suspicious events. Those features are only available with Respondus Monitor, the automated proctoring solution.

The RLDB Instructor Live Proctoring option is supported on Windows, MacOS and the new Chromebook Extension. LockDown Browser for iPad doesn’t directly support the Instructor Live Proctoring option (the iPad assessment mode doesn’t allow multiple apps to run simultaneously), but similar results can be achieved if a second device is used.

Want more information?

Instructor Live Proctoring with LockDown Browser (video)
About LockDown Browser - Instructor Live Proctoring
Instructor Live Proctoring Quick Start Guide
RLDB Instructor Live Proctoring FAQ
Brightspace Tip #200: Respondus LockDown Browser / Respondus Monitor

View all the Brightspace training recaps
Instructors Quick Start Tutorial
Continuous Delivery release notes
Brightspace Known Issues
Request a sandbox course
Sign-up for Brightspace training sessions
You can find Brightspace help at D2L's website.
Join the Brightspace Community.
Try these Brightspace How-To documents.
Visit our Brightspace FAQs for additional Brightspace information
or schedule a one-on-one session, email, or
call Janice Florent: (504) 520-7418.

Note: Are you doing something innovative in Brightspace or perhaps you've discovered a handy tip? Share how you are using Brightspace in your teaching and learning in The Orange Room.

The 'Observed in Person' assignment submission type is for situations where a learner needs to present something or demonstrate something and an instructor needs to confirm that it was completed. This submission type allows instructors to create assignments for a wide variety of activities and provide evaluation and feedback for these assignments in Brightspace.

assignment submission types

The ‘Observed in Person’ submission type is meant for scoring and evaluation of a learning activity such as demonstration, presentation, lab test, etc.

There are four submission types to choose from when creating an Assignment in Brightspace. The submission types are:

  • File submission: learners upload and submit a file to the assignment.
  • Text submission: learners post text, image, or a link to their work in a text box within the assignment.
  • On paper submission: learners submit a physical copy of their assignment directly to their instructor. No file upload is required to complete the assignment.
  • Observed in person: allows instructors to observe learners as they perform tasks, such as in-class presentations, and evaluate the observation. No file upload is required to complete the assignment.

There are three options to mark an ‘Observed in Person’ assignment as completed:

  • Automatically on due date – Brightspace marks the assignment as completed on a pre-set date.
  • Automatically on evaluation – Brightspace automatically marks the assignment as complete when the instructor evaluates the assignment using any of the assessment methods in Brightspace.
  • Manually by Learner – The student will mark the assignment as complete.

The assignment can be assessed using text/audio/video feedback, Brightspace interactive rubric, or by assigning a grade.

Want more information?

What is an ‘Observed in Person’ assignment?
Create assignments and assess submissions
Brightspace Tip #142: Simplify Assignment Collection
Brightspace Tip #204: Interactive Rubrics
Brightspace Tip #102: Video Notes
Brightspace Tip #157: Video Notes - Closed Captions
Brightspace Tip #169: Video Notes - Recording Limit

View all the Brightspace training recaps
Instructors Quick Start Tutorial
Continuous Delivery release notes
Brightspace Known Issues
Request a sandbox course
Sign-up for Brightspace training sessions
You can find Brightspace help at D2L's website.
Join the Brightspace Community.
Try these Brightspace How-To documents.
Visit our Brightspace FAQs for additional Brightspace information
or schedule a one-on-one session, email, or
call Janice Florent: (504) 520-7418.

Note: Are you doing something innovative in Brightspace or perhaps you've discovered a handy tip? Share how you are using Brightspace in your teaching and learning in The Orange Room.

interstate highway with rubrics road signs

Brightspace interactive rubrics help instructors:

  • Increase Efficiency - Rubrics are built into the grading workflow. Rubrics click-and-score simplicity saves time.
  • Provide Consistent and Quality Feedback - Rubrics enable instructors to provide consistent evaluation and contextual feedback to students.
  • Promote 21st Century Skills - Rubrics make it easier to assign essay questions, individual and group assignments, and discussion forums as assessment activities which foster critical thinking and collaboration.

Rubrics allow instructors to establish set criteria for grading assignments; instructors can attach rubrics to submission folders so that the criteria are available to students before they submit their assignment.

Weighted analytic rubric creation example
Weighted analytic rubric creation example

Rubrics contain criteria that list the attributes on which an assignment will be assessed and levels that list the standards each criterion must meet. A specific grade or score is usually assigned to each level. In Brightspace, you can use a rubric to calculate scores for multiple criteria to determine an overall score for an assignment.

example of grading with a rubric
Grade using a rubric example

Rubrics can be used to display the number of points students were awarded for each criterion after the assignment is graded and rubrics can also be used to provide customized feedback.

Instructors can choose to have the rubrics visible to students at any time, only after grading has been completed, or not shown to the students at all.

If you are looking for more information on rubrics, check out these resources:

NOTE: The Brightspace Rubrics tool is different from Turnitin Rubrics.

Follow these steps to do it.

To create a rubric you should:

  1. On the navbar, click Course Admin.
  2. Click Rubrics.
  3. On the Rubrics page, click New Rubric.
  4. Enter a name for your rubric.
  5. Change the status of your rubric, if necessary.
  6. Choose the rubric Type and Scoring method.
  7. Enter the criteria, levels, criteria/level details, and initial feedback for your rubric.
  8. Enter details for the Overall Score feedback.
  9. Click Options and choose the options for your rubric.
  10. Click Close.

Note: Rubric changes are automatically saved.

To attach a rubric to a Discussion topic:

Note: You must first create a rubric and set its Status to Published.

  1. On the course home page, click Discussions in the NavBar.
  2. Click Edit Topic from the context menu of the discussion topic you want to attach a rubric to.
  3. In the Assessments tab, click Add Rubric.
  4. Select the check box for the rubric you want to attach to the discussion topic.
  5. Click Add Selected.
  6. Click Save and Close.

To attach a rubric to an Assignment:

Note: You must first create a rubric and set its Status to Published.

  1. On the NavBar, click Activities and then click on Assignments.
  2. Click Edit Assignment from the context menu of the assignment you want to attach a rubric to.
  3. In the Properties tab, click Add Rubric.
  4. Select the check box for the rubric you want to attach to the assignment.
  5. Click Add Selected.
  6. Click Save and Close.

Want more information?

Rubrics Tool Quick Reference Guide (pdf)
Create a Rubric
Create an Analytic Rubric (video)
Create a Weighted Rubric (video)
Create a Holistic Rubric (video)
Add a Rubric to an Existing Activity (video)
Attach a Rubric to a Discussion Topic (video)
Grading with a Rubric
Add Feedback and Evaluations to Assignments (video)
Rubrics FAQ

View all the Brightspace training recaps
Instructors Quick Start Tutorial
Continuous Delivery release notes
Brightspace Known Issues
Request a sandbox course
Sign-up for Brightspace training sessions
You can find Brightspace help at D2L's website.
Join the Brightspace Community.
Try these Brightspace How-To documents.
Visit our Brightspace FAQs for additional Brightspace information
or schedule a one-on-one session, email, or
call Janice Florent: (504) 520-7418.

Note: Are you doing something innovative in Brightspace or perhaps you've discovered a handy tip? Share how you are using Brightspace in your teaching and learning in The Orange Room.

Image credit: "Rubric Highway" by Jen Hegna licensed under CC BY-NC-SA 2.0

update

D2L (the company that owns Brightspace) uses Continuous Delivery to update our Brightspace system. The Continuous Delivery model gives us regular monthly updates allowing for incremental and easily integrated changes with no downtime required for our Brightspace system.

Our Continuous Delivery update occurs on the 4th Thursday of each month. D2L provides release notes to help users stay up-to-date with the changes.

Here are a few updates in the January 2021/20.21.1 release that were added to our system this month:

1) Assignments - Multiple rubric support in the consistent evaluation experience

This feature is a continuation of the Assignments - Consistent evaluation experience | New feature that was introduced in the December 2020/20.20.12 release.

If an assignment includes multiple rubrics, they are all visible to instructors using the Consistent evaluation experience. Instructors are also able to choose the default scoring rubric. Previously, the consistent evaluation experience would only display one rubric on the assignment evaluation screen.
For additional information on the Consistent evaluation experience, visit Brightspace Community.

2) Brightspace Learning Environment – Supported browsers

Brightspace Learning Environment's browser version check now displays warnings for browsers earlier than the following versions:

  • Google® Chrome™ 85 (released Aug 2020)
  • Apple® Safari® 13 (released in September 2019)
  • Mobile Apple Safari 12 (released in October 2018)
  • Mozilla® Firefox® 80 (released August 2020)
  • Chromium-based Microsoft® Edge 85 (released August 2020)
  • All versions of Microsoft Edge Legacy (in preparation for Microsoft’s end-of-support in March 2021)

You can access Brightspace with older versions of the supported browsers; however, D2L does not test Brightspace against these older versions. This means you may encounter unexpected user interface behavior and appearance or broken and unsupported functionality. In most cases, D2L does not fix software defects experienced in unsupported browser versions. For an optimal experience that offers better performance, accessibility, and security, D2L strongly recommends that all users access Brightspace with a supported browser.

3) Course Export - Export file upload attribute for written response questions

Written Response quiz questions that allow for file upload can now be imported and exported from Brightspace Learning Environment. With the ability to import and export these questions users can share them with users from other instances of Brightspace.

If you are interested in getting more information about these and all the January Continuous Delivery updates, refer to the Brightspace Platform January 2021/20.21.1 Release Notes.

Additionally, refer to the Brightspace Release Notes for Continuous Delivery Releases, for details about current, past, and to preview upcoming continuous delivery updates.

Want more information?

View current, past, and preview upcoming Continuous Delivery release notes
View all the Brightspace training recaps
Instructors Quick Start Tutorial
Brightspace Known Issues
Request a sandbox course
Sign-up for Brightspace training sessions
You can find Brightspace help at D2L's website.
Join the Brightspace Community.
Try these Brightspace How-To documents.
Visit our Brightspace FAQs for additional Brightspace information
or schedule a one-on-one session, email, or
call Janice Florent: (504) 520-7418.

Note: Are you doing something innovative in Brightspace or perhaps you've discovered a handy tip? Share how you are using Brightspace in your teaching and learning in The Orange Room.

Image credit: image by geralt from Pixabay

female student looking at computer screen

Giving students timely, useful feedback can greatly enhance learning and improve student achievement. ICYMI, read my Give Students Feedback That Helps Them Learn blog post for information on providing better feedback.

Are you looking for ways to simplify grading and for providing feedback on assignment submissions? These Brightspace tools can help:

Annotations Tool
The Annotations tool allows instructors to provide feedback directly in submitted assignments. Feedback can be provided as text highlighting, freehand drawing, text annotation, and note annotation.

The use of a stylus is not required when using the Annotations tool. However, it could help to speed up the process of marking up the assignment submissions.

For more information, refer to this how to use the Annotations tool blog post.

NOTE: The Assignment Grader app allows instructors to make annotations on assignments. The Assignment Grader app is no longer available for download to new users. Existing users may continue to use the app if they have it installed on their device, but no further updates or support will be made available by D2L and the app is scheduled to be removed from app stores. As an alternative mobile grading option, D2L recommends using the Quick Eval tool.

Rubrics
Rubrics allow instructors to establish set criteria for grading assignments. Using the rubrics click-and-score simplicity saves time when grading. With a rubric you can provide consistent evaluation and contextual feedback to students. You can add additional personal feedback to each criterion, expanding on why you chose that level and what additional work would be required to improve on it.

For more information, refer to this using interactive rubrics in Brightspace blog post.

Video Notes
Audio and video feedback can provide richer and more detailed feedback than may be possible through written comments. Visual or audio feedback also provides a more personal way of giving feedback. You have the option to provide audio and/or video feedback in Brightspace. Use Video Notes to provide short, video-based feedback, comments, or instructions.

Here are a few blog posts with more information:

Quick Eval
Quick Eval allows evaluators to see a list of unevaluated learner submissions from all their courses. Submissions from Assignments, Discussions, and Quizzes are displayed in one location to improve efficiency when locating work that requires evaluation and providing feedback to learners.

For more information, refer to this blog post on using Quick Eval.

The Orange Room

Have you visited The Orange Room (Brightspace Educator Share Showcase)? The Orange Room is a community where educators can learn from each other about efficient and innovative ways to use Brightspace. Several of your colleagues shared tips and suggestions already. Here are two contributions that may be of interest to you:

Want more information?

Brightspace Tip #142: Simplify Assignment Collection
Brightspace Tip #143: Annotate Assignment Submissions
Brightspace Tip #167: Interactive Rubrics
Brightspace Tip #102: Video Notes
Brightspace Tip #157: Video Notes – Closed Captions
Brightspace Tip #169: Video Notes – Recording Limit
Brightspace Tip #120: Quick Eval

View all the Brightspace training recaps
Instructors Quick Start Tutorial
Continuous Delivery release notes
Brightspace Known Issues
Request a sandbox course
Sign-up for Brightspace training sessions
You can find Brightspace help at D2L's website.
Join the Brightspace Community.
Try these Brightspace How-To documents.
Visit our Brightspace FAQs for additional Brightspace information
or schedule a one-on-one session, email, or
call Janice Florent: (504) 520-7418.

Note: Are you doing something innovative in Brightspace or perhaps you've discovered a handy tip? Share how you are using Brightspace in your teaching and learning in The Orange Room.

Photo credit: photo by #WOCinTech Chat is licensed under CC BY 2.0

person holding a mobile phone

Students can control how they receive information about course activity through their notification preferences. They can customize which actions in Brightspace will trigger a notification.

The Notification tool allows users to:

  • Subscribe to a summary of activity for each course and receive a daily email.
  • Specify your preferred email address and mobile number for instant notifications and announcements.
  • Receive instant notifications about course activity, such as edited content, new discussion posts, assignments, grades, course announcements and upcoming quizzes.

Instructors can help their students to keep up with their coursework by encouraging them to setup their notification preferences. Additionally, there is a Getting Started for Learners Video Playlist that students should watch to get familiar with Brightspace.

Follow these steps to do it.

To customize notification preferences, users should:

  1. From your Personal Menu (located in the Minibar), select Notifications.
  2. select notifications from personal menu

  3. Select the email address and/or register the mobile phone number to which notifications will be sent.
  4. Select the course activities for which you would like to receive notifications.
  5. Click the Save button at the bottom of the screen.

Want more information?

Navigate Brightspace - Notifications - Learner (video)
Getting Started for Learners Video Playlist

View all the Brightspace training recaps
Instructors Quick Start Tutorial
Continuous Delivery release notes
Brightspace Known Issues
Request a sandbox course
Sign-up for Brightspace training sessions
You can find Brightspace help at D2L's website.
Join the Brightspace Community.
Try these Brightspace How-To documents.
Visit our Brightspace FAQs for additional Brightspace information
or schedule a one-on-one session, email, or
call Janice Florent: (504) 520-7418.

Note: Are you doing something innovative in Brightspace or perhaps you've discovered a handy tip? Share how you are using Brightspace in your teaching and learning in The Orange Room.

Image credit: image by tranmautritam from Pixabay

Respondus LockDown Browser image

Respondus LockDown Browser (RLDB) is a customized browser that increases the security of test delivery in Brightspace. When students use Respondus LockDown Browser to access an exam, they are unable to print, copy, go to another URL, or access other applications. Once an assessment is started, students are locked into it until it's submitted for grading.

Respondus Monitor image

Respondus Monitor builds upon the power of LockDown Browser, using a student's webcam and industry-leading video analytics to discourage cheating during online exams.

RLDB / Respondus Monitor Resources

You may find these resources helpful if you are planning to use Responus LockDown Browser / Respondus Monitor:

Practice Quiz

If you are using Respondus LockDown Browser (RLDB) / Respondus Monitor for online exams, you should require students to take a practice quiz before the first test that requires RLDB/Respondus Monitor. The practice quiz should have a few questions in it to simulate what the student would experience when taking an actual exam. By taking the practice quiz the student can ensure that their computer, webcam and internet are working properly. Allow unlimited attempts on the practice quiz, so students can take it whenever their device or networking environment changes. Here are some tips on how to run a successful practice quiz.

Respondus 24/7 Live Chat

24/7 Live Chat is available for students who may experience problems while taking an exam that requires RLDB/Respondus Monitor. Students can access built-in troubleshooting and live chat from within Respondus Monitor if they encounter a technical problem when taking an exam that requires RLDB/Respondus Monitor. You should provide your students with the link to the RLDB/Respondus Monitor Student Resources which includes information on how to access the 24/7 Live Chat (by Respondus Tech Support).

RLDB Settings in Copied Courses

If you copy a course that has tests/exams with RLDB enabled, you will have to access the LockDown Browser Dashboard once after the copy to update the RLDB settings in the destination course. This has to be done before students will be able to take exams that require RLDB.

Want more information?

For Instructors:

For Students:

View all the Brightspace training recaps
Instructors Quick Start Tutorial
Continuous Delivery release notes
Brightspace Known Issues
Request a sandbox course
Sign-up for Brightspace training sessions
You can find Brightspace help at D2L's website.
Join the Brightspace Community.
Try these Brightspace How-To documents.
Visit our Brightspace FAQs for additional Brightspace information
or schedule a one-on-one session, email, or
call Janice Florent: (504) 520-7418.

Note: Are you doing something innovative in Brightspace or perhaps you've discovered a handy tip? Share how you are using Brightspace in your teaching and learning in The Orange Room.

Updated: 9/4/2021