During the course of the ”Exploring Adversarial Machine Learning" I’ve had issues with the final assessment particularly extraction. I don’t have access too and can’t download the food101 dataset from HF. Its wasted an enormous amount of GPU time. I’ve passed all assessment except extraction.
Could someone hard reset my training course. I know at one point I had the food101 dataset. I also need 8+ hours to figure this assessment out.
Would it be possible to extend some more time for me to complete the course?
@TomNVIDIA, I need help on this. Is there someone who can help? I can score 9/18 points on the extraction assessment, but a passing grade for the cert is 90. My total score is an 85.
Exploring Adversarial Machine Learning - Extraction Assessment, can’t get 18/18
Results before training
AUC with victim_model labels: 0.527640442247076
Results after training
AUC with victim_model labels: 0.011408182530920491
AUC with proxy_model labels: 0.015216000000000004
Thank you, the issue is that 2 files are supposed to be created
from the lab, "Your submission for this assessment will be two files:
dict with the counts of the extracted labels with the following format:
counts_dict= {'pizza': ###, 'hotdog': ###}
where ### is replaced with the correct values. Remember that we wish to build a pizza/hotdog classifier, so we should be looking for the highest output between only those two classes returned by the victim_model"
Sorry for the frustration with loading the food101 dataset. It looks like huggingface-hub made an update in the past month or so that required a non-backwards-compatible update to datasets on the huggingface site, meaning older versions of huggingface/datasets can’t load those datasets (as you experienced). See, for instance, discussion here.
Unfortunately, HuggingFace unilaterally implementing changes like this is beyond our control. We’re looking into a more durable fix, but in the meantime, please try the following. Inside the extraction assessment notebook:
Restart your kernel (Kernel → Restart Kernel)
At the very top of the notebook, before any other code, please create a new code execution cell (click the top cell, hit ESC, then type the letter ‘a’ once) then copy and paste the following code into that new cell: !pip install --upgrade datasets huggingface-hub pillow and run it; this will update some key libraries to the current version that can read the updated datasets. It will give some warning messages and an error about pip’s dependency resolver; please ignore them.
Restart your kernel again (Kernel → Restart Kernel) and comment out the !pip install ... command
Run the notebook as normal.
This should allow you to load and work with the food101 dataset normally.
Please be aware that the entire dataset loading, filtering, and preprocessing step may take several minutes; if it appears to hang – particularly on the map step – give it a full five minutes to resolve before worrying.
Doing this blunt-instrument update with pip may cause issues with other dependencies elsewhere in the course, so we recommend that you do this last – it sounds like this is the only thing you have left to do so it shouldn’t be a big issue for you. In the worst case, stopping and restarting your image from the DLI course page will revert any changes caused by using pip.
Again, apologies for the frustration, and thank you for bringing the issue to our attention.
@TomNVIDIA, I’m very close now to solving the extraction assessment. DLI did fix issues with the dataset and a file that needed to be saves. Can you please extend my GPU? Any help would be great.
@rharang can I please get 8-16 hours of additional GPU to complete the assessment? I spent so much time trying to create a workaround and am now out of GPU. This is my last assessment I need to pass for this certification.
I did manage to complete the poisoning assessment after, and the evasion assessment as well. So it seems the dependencies aren’t affecting those two at least. And hopefully the others are fine as well.
UPDATE: For those coming later to this thread, the dependency upgrade issue resolved in the solution is now addressed directly in the notebook. Now you don’t need to do anything other than follow the provided cells that upgrade and restore the dependency configuration.