https://github.com/astral-sh/uv/issues/1980
https://github.com/reflex-dev/reflex/issues/2335
https://github.com/x64dbg/x64dbg/issues/3050
https://github.com/x64dbg/x64dbg/issues/3137
For the last 4-5 days, I’ve been spending a large amount of time investigating a bug and figuring out its fix. This meant, diving into rabbit holes, going deeper and deeper and surrounding myself with documentation and online discussion forums. I’m briefly outlining the process here mostly because I’m pretty proud of what I did.
Link to the Github Issue
So I was going through the open source repository of the Python web framework Reflex and I saw that they had an issue that was preventing them from using Reflex in Windows systems with Python 3.12. The issue was that whenever they were using Reflex in this environment and they leveraged the hot reload feature of the tool, the frontend server just dies. In other words, when you make a change in one of the files, you expect the running servers to detect it and reload the website with the new changes. You can see in the logs that the change has indeed been detected and they have compiled the new version of the website but for some reason the frontend server is not alive anymore. An early clue that I caught was that the logs were showing the character combination ‘[?025h’ whenever this was happening. I had also noticed that whenever I stopped Reflex intentionally with a Ctrl+C, this character combination was getting outputted as well. So the first thought was, okay somehow a Ctrl+C from somewhere was affecting the frontend server.
The developer had mentioned in the issue discussion that the bug is connected to one of their upstream dependency ‘uvicorn’. Uvicorn is an ASGI web server in Python and Reflex uses Uvicorn as their backend server in development mode. Uvicorn has a reload functionality that detects the changes in the server directory and updates the server accordingly. I tested out the uvicorn server on its own and the reload functionality was working perfectly. The next thing I did was go through the reflex code and see how they were using the uvicorn server inside it. I also studied how they were spawning the frontend server as well. With this information, I created a very small mock reflex, that was using uvciorn server and a mock frontend server (a while loop printing the word “Frontend”) just the way the original reflex implemented it. In short, the frontend server was initiated as a python subprocess and the uvicorn server ran in the main thread. This mock version took away a lot of the complexities and reduced it down to simple elements so that I can zone in on where the issue is. And sure enough, when I ran it, a reload of the uvicorn server killed my mock frontend.
# main.py
import subprocess
import uvicorn
process = subprocess.Popen(["python", "frontend.py"], shell=True)
print("Starting uvicorn")
uvicorn.run(
app="uvicorn_main:app",
port=5000,
log_level="debug",
reload=True)
# frontend.py
import time
if __name__ == "__main__":
while True:
print("frontend")
time.sleep(5)
# uvicorn_main.py
async def app(scope, receive, send):
assert scope['type'] == 'http'
await send({
'type': 'http.response.start',
'status': 200,
'headers': [
[b'content-type', b'text/plain'],
],
})
await send({
'type': 'http.response.body',
'body': b'Hello, world!',
})
Link to the Uvicorn Discussion
At this point, I had started using a Process Explorer to see the process hierarchy in order to try and figure out how a reload process in the main thread could affect a subprocess. I also started studying the code in uvicorn to see how they carried out their reload process. I noticed that on detecting file changes they kill their server process (a child process in the hierarchy) and they start it over again. And an even more interesting bit was that they were using a CTRL_C_EVENT signal to close the server. Since I was on the lookout for Ctrl+C from the beginning, I started tracing out the changes to this piece of code through their changelog. I noticed that earlier they were using a Process.terminate() function and they changed it to the Ctrl+C event a couple of versions back. This again corroborated our issue as I noticed that Python 3.11 was using a uvicorn version before this change whereas Python 3.12 was using the one after.
Now, I was getting pretty sure that it is this line of code that was affecting my frontend server and I did a couple of tests to confirm that all the versions before this change were working perfectly. Now the question was, why would uvicorn sending a Ctrl+C to one of its child affect a subprocess that was initiated by its ancestor. That made absolutely no sense to me. So I went into more deep dives on the Ctrl+C event.
After a lot of internet searches, I reached these two links
- https://bugs.python.org/issue42962
- https://github.com/microsoft/terminal/issues/335
Turns out, in windows when you’re sending Ctrl+C events using the os.kill function, it is meant to be sent to process groups and not individual processes. If you do send it to an individual process, it just defaults to the root process group (which is the console that started reflex) and hence all its children. That solves the mystery of how uvicorn was managing to reach the frontend subprocess.
Now that I know the reason behind the bug, how do I fix this? Getting uvicorn to fix this bug was one option, but that would probably take time and in the meanwhile Reflex could use a workaround. Back to more rabbit holes.
As I was going through the Python subprocess documentation, I saw that they had a creationflags parameter that took in an attribute CREATE_NEW_PROCESS_GROUP. That leads me to the idea, what if I start the frontend subprocess as a new process group? Would the uvicorn Ctrl+C event still reach it? I got down to testing it out, and to my happiness I saw that uvicorn was not able to stop the frontend anymore. And then, to my sadness I saw that I was not able to stop the frontend anymore either. Basically, the hot reload issue was fixed but now when I tried to quit Reflex as a whole with a Ctrl+C, even though uvicorn was exiting properly the frontend was just ignoring me. What’s up with that?
I started going through the documentation for the python subprocess creation flag as well as the CreateProcess windows function that was happening behind the scenes. The story being told by both these parties were slightly different. The Python documentation claims that when you create a subprocess with the CREATE_NEW_PROCESS_GROUP flag, you can send it a Ctrl+C event using the os.kill function. However, the CreateProcess Windows function tells you that one of the first things they do when you create a new process group is that they turn off the Ctrl+C handler for it. Which means the new process group does not respond Ctrl+C signals anymore. At this point I’m very confused about how this whole Ctrl + C event and process groups were intended to be in the first place. I did test this information out with my Mock Reflex and it checked out. The Ctrl+C handler was indeed turned off for the new process group and that was the reason why uvicorn’s buggy Ctrl+C was not affecting the frontend. Because the frontend, does not see Ctrl+C at all anymore.
So what do I do with this? Well, maybe the frontend does not respond to Ctrl+C anymore, but it does respond to other signals such as Ctrl+Break event and SIGTERM. After discussions with the developers of Reflex, we decided that as a workaround once uvicorn exits Reflex will send a SIGTERM to the frontend.
That brings me to the end of my long drawn out investigation on understanding a bug and providing a solution for it. The deeper you go into the rabbit hole the more tangled you get in the details, but the satisfaction of finding answers is very rewarding.
High level Overview of Steps to Unpack UPX dynamically
- Find the original entry point of the packed file
- Run Scylla to extract the payload out
- Turn on the IMAGE_FILE_RELOCS_STRIPPED flag inside the extracted PE file
Finding OEP
There are a lot of different ways you can go about doing this, and it would depend heaviy on the packer used and the complexity of it. For a simple hello world program that was packed using UPX the following was the easiest way in which I could find the OEP.
- Break at the entry point
- Observe that the instruction at the entry point is a push instruction. Step over that instruction
- Set a memory breakpoint that will get triggered on accessing the location to which esp/rsp is pointing
- Run the program until the memory breakpoint is hit. This would likely be after a pop instruction (this would be the counterpart to the initial push instruction)
- Observe the dissassembly to find a jump instruction in the next few lines. The jump address is likely to be the OEP
- If you allow the program to jump to the OEP, you can observe the pattern of instructions to see that it resembles a new PE executable. If you are doing this as an exercise you would likely have the original unpacked file. You could compare the bytes of the original executable against the bytes at the OEP to confirm your strategy
Running Scylla
Running scylla is fairly well documented. If it is run as a plugin of a debugger, the module start address, size and such would already be filled. All you would need to modify would be the OEP. With the right OEP Scylla should be able to get the right import table. There might be unkown chunks in the imports that scylla had obtained. For a hello world program removing the unknown chunk was enough to get it working. More complicated programs might require more steps to resolve the imports. Dump the payload using Scylla and fix the Import Table using the Rebuild function of Scylla.
Patching the PE file
After all of this, even though the Scylla dumped file was very similar to the original payload it was crashing on execution. To fix this, open a PE editing tool such as PEBear, or CFF Explorer and turn on the IMAGE_FILE_RELOCS_STRIPPED flag.
Creating YARA rules to match against malware files using Machine Learning Model. This was for a course project, where we were supposed to come up with some improvement on an existing paper. So we decided to see if we can get AutoYara up and running and then maybe do some experiments on it. The link to the original work is... Here. Oh and by 'we' that I use throughout this post, I mean me and my friend Soumya. We also took the guidance of Dr.Marcus Botacin who was gracious enough to give his two cents about our issues, even though it was not his course or anything.
First of all the entire original project by the authors is in Java and we decided to reimplement this in Python. The idea was that once in Python we can make use of all the existing Machine Learning libraries and mess around with it. Dr Botacin did suggest us that a possible imporvement would be to reimplement the project in C as that could lead to an increase in general efficiency. Due to lack of time, we decided not to go down that path, and we put the C idea in the board for future work. At this point maybe a small high level run down on how AutoYARA works might be helpful.
AutoYARA creates YARA rules provided a small number of input files, maybe even 3 or 4. Now, ideally these files should be from the same family so that the system would be able to identify common unique patterns and use that to build the YARA rule. The first phase that it goes through is to extract important ngrams from these files. How do you identify what's important and what's not. The authors have created a list of frequently seen ngrams in the malware domain and goodware domain, by using datasets like Ember. This information is stored in Bloom Filters and these bloom filters are used during the YARA generation to extract out the important ngrams from the input malware files. Other filteration steps are also done but by the end of it you have a set of ngrams. The second phase is to run a coclustering algorithm on the ngrams and the input malware files(samples). A co-clustering algorithm creates clusters by exploring the relationship between two entities in our case ngrams and samples. So for example, a cluster would have a set of ngrams and samples in it, which kind of indicate that this set of samples use or have this set of ngrams in it. AutoYARA uses Spectral CoClustering with Variational Based Gaussian Mixture Model(VBGMM) for this purpose. The advantage of VBGMM is that you do not have to specify the cluster size to the algirthm which is good since we don't really know how many clsuters should be formed. Now, once you have these clusters you can create the YARA rule from it. The intuition is that, if a future file falls into any of these clusters, then it means the YARA rule should match. So the YARA rule would have a similar logic to , "if the file belongs to cluster 1 or belongs to cluster 2 or belongs to cluster 3" then the YARA Rule matches. Now, belonging to a cluster is determined by the ngrams. So the rule would be like "If the file has ngrams (1,2 and 3) or ngrams (5,6 and 7) or (8 and 9)" then the YARA matches. Notice how the rule is a set of AND conditions connected by OR condition, so basically a Disjunction of Conjunction Statements. Another important point is that a file do not have to contain all of ngrams (1,2 and 3). It would be more like, if the file contains 2 among the three, then we can consider it to be part of the cluster and hence decide that the YARA rule matches.
On to the implementation. The first major trouble was that the ngrams themselves. The authors use really large ngrams to the order of 1024 bytes. Extracting such large ngrams from a dataset and creating bloom filters ourselves is a difficult task in itself. Kilograms is the technique that the authors used to extract the top k large ngrams from a dataset efficiently with limited memory. I believe KiloGrams was created by the same team as well. Now we did not have the time to go into reimplementing KiloGrams from scratch too, and hence wanted to reuse the bloom filers that the authors have made publically available in Java. Yes, the dataset from which the bloom filters were created is a little outdated (2017) but it still might serve our purpose for experimenting. So we created a bridge between our Python Application and a Java application that uses the author's Java implementation of ngram extraction and filteration. Once we have the filtered set of ngrams that has passed through the Blooom filters we take over and do the rest in python.
The next major hurdle was that Spectral Coclustering package available in libraries like SciKit uses KMeans and do not use methods like VBGMM. This means that we would have to specify the cluster size for it to work. Alright, let's specify the cluster sizes and maybe we can mess around with the cluster sizes and see its influence on the results.
With all these compromises that we had to go through due to the limited time frame, we implemented AutoYARA and managed to get the YARA rules out. We tested it on malware from the same family downloaded from VX-underground and the results were to be honest, not that great. The main conclusion we observed was that the cluster sizes and the ngram sizes mattered a lot. For each malware family there was atleast one ngram size and cluster size combination that performed extremely well. But the task was in identifying this. The cluster size issue would probably be taken care of by employing VBGMM instead of KMeans which means we would not have to specify the cluster size. The authors of AutoYARA evaluates the effectiveness of the rules generated across multiple ngram sizes in some way that we were not able to cover yet. So the lackluster results was more of an issue of the compromises we had to make and not the technique itself.
So yes, there's a lot of future work to be done in this before calling it a closed project, so watch out for that. The source to our Pythonic limtied(atleast for now) implementation is here.
On a path to understand the widely famed power of Awk. Hope to conquer it one day, just for the sake of it. Sed is the next one on the bucket list
Most basic use case
Say you want to print out the second word from each line.
awk '{print $2}' input_file
Awk follows a <pattern> <action> format. For each line, awk will apply the pattern and see if it matches. If it does match, it'll perform the action. Now, you can omit either pattern or action but atleast one should be present. If action is omitted, the default action is to just print the line out. If pattern is omitted, the action would be applied to all the lines.
In the above example of printing out the second word, there is no pattern but only action. $1 corresponds to the first word, $2 to the second and so on. $0 corresponds to the whole line.
awk '/^A/ {print $2}' input_file
This is similar to printing out the second word, but instead of in each lines it just prints it out on the lines that starts with the letter A. The regex pattern in the awk command enclosed within the slashes acts as the pattern to be matched against.
Awk can handle a whole set of pattern action tuples instead of just one. So if you write out
awk '<pattern> <action>
<pattern> <action>
<pattern> <action>' input_file
for each line in the file, all patterns would be matched up one after the other. If a pattern matches, its correpsonding action would be performed on the line.
Variables makes it awesome!
You can use variables inside the awk command, making it a whole programming language capable of performing complex logic. There are variables you can define and there are some in-built variables as well.
Let's say you want to print out the second word of the second line that starts with the letter A.
awk '/^A/ {n++; if(n==2) print $2;}' input_file
A couple of things about the above command. First thing that's worth noticing is that you can concatenate multiple commands inside the action part of awk. They'll be executed one after the other just like a normal programming language. Another thing is that you can just start using a variable and it'll be initialized to 0 by default. So the first time awk encounters a line starting with A, it'll initialize the variable n with the value 0, increment it and then check it against the if condition. The second time the pattern matches, the value of n will increment from 1 to 2, which makes the if condition true causing it to print the second word of that line.
Still under construction. More to come...
So, my initial plan to create this blog was to build it from scratch, brick by brick, maybe even make a content management system on the way. However, due to a lack of time and a general decision in life to stop rebuilding the wheel every single time, I decided to use something someone wise has already built.
I did intially go for a framework called CMSjs which I found here but for some reason I did not like the way it worked. Finally I settled down to Jekyll and decided to get version 1 of the blog up and running as soon as possible instead of trying to find the perfect tool out there. Barry Clark, a Web Developer from NY, has this very straight-forward article to set up a minimal blog by forking a starting-point that he has created. Following it was a breeze, and before I knew it I had a pretty decent looking blog in my hands.
Subpage with different posts
Once I had the basic blog ready, I decided to post about 24 summaries that I had made about ML Based Malware Detection Systems over the past 6 months. However, I did not want to make my homepage messy by having all these summaries in there. What I wanted was a brief post in the homepage "sid-38.github.io" that will lead you to a sub-page "sid-38.github.io/mlcyber" which would contain all the summaries. Alright, so how to create pages in Jekyll?
You can just create a Markdown file with the layout as "page" and that would work pretty well. However, I did not want to create one HUGE markdown file that contains 24 articles inside it. That would be a really long file, won't it? What I wanted was for my MLCyber page to fetch all the posts and display it just like my homepage does.
How Jekyll works normally is, it'll fetch the files from the _posts directory when you use the variable "site.posts". This allows you to iterate through it and display it in your html page. So I figured if I create a subdirectory called "MLCyber", create an html file "MLCyber.html" inside of it along with a subdirectory _posts, maybe it could work.
Root
_posts
MLCyber
mlcyber.html
_posts
post1.md
post2.md
Maybe, if I use "site.posts" inside of mlcyber.html it'll only see post1 and post2? Well, that did not work but I was pretty close. After a number of iterations and even going into Jekyll Collections, none of it worked.
How to get posts from subdirectories
Jekyll has a concept of categories. Utilizing categories, it is possible to maintain the structure shown above and get the render we were looking for. For the mlcyber posts, in the frontmatter of jekyll, add "category: mlcyber". And to iterate over just the posts in the category mlcyber, inside mlcyber.html do the following.
{% for post in site.categories.mlcyber %}
I saw a number of articles that said if the posts are inside a subdirectory it should by default be assigned a category based on the parent directory name. This would have been extremely convenient, but it did not work for me when I tried it out. I also tried putting the subdirectory mlcyber which contained post1.md and post2.md, inside the root _posts. Even that combination did not assign the category automatically. Well, maybe I made some typo somewhere or something, I don't know.
Alright, that's step 1 done. Now to fix the next issue. The homepage now shows every post, including the mlcyber posts. How do I display just the posts that did not have a category assigned to them?
{% if post.categories.size == 0 %}
Yup, that pretty much got the blog to where I wanted it to be. Now, onwards and upwards to more posts.
For my course in TAMU - ML Based Cyber Defense under Dr. Botacin, throughout the semester I had to make summaries for different research papers in the domain. During each class, a student had to give a seminar on a research paper and the others had to prepare a summary on the paper as well as the discussion that followed the seminar. I am uploading all of the summaries that I created into this blog. But fair warning, finishing the summaries as soon as possible for the sake of submission, in the midst of all other course work doesn't always lead to the best quality writing.
Still, for what it's worth the summaries would be available here
For the course "ML Based CyberDefenses" under Dr.Botacin we had a very interesting and hands-on in-class competition. You form teams, and then each team come up with a malware detection system as well as generate some adversarial samples (samples that are designed to evade the ML detection models) using existing malware files. The adversarial samples that our "competitors" came up with would be put against our detection system, and vice versa to decide how much points each team scores. Luckily, I was able to find a team that valued team-building as importantly as actual work leading to a lot of fun memories. Big Shout out to SidBav, Soumya and Veronika. And to Zoe our team-mascot (Not to brag, but I won her from a claw machine first try)
Defend
Initial Dataset and Feature Extractor
For the defensive part, the basic strategy was a trial and error through different ML Models. For the dataset, we went for the EmberDataset for the basic layer with a plan to add on to it with more recent data or data that is tailored for the competition. Ember has its own feature extractor that was throwing some errors when we tried to get it up and running. We debugged through the error to find that due to updates in LIEF library, a library that can be used to read and modify executable files, a certain attribute was not in the format that Ember Feature Extractor was expecting. If I remember correctly, it was the 'entry' attribute that needed to be enclosed in an extra pair of square brackets. Ember by default uses a LightGBM model and that's what we went ahead with for the basic version. Later we switched to a Random Forest Classifier model and submitted that for the first deliverable. After this blind submission, the professor released the samples he was testing our model against. In order to meet some criteria the competition had regarding False Positive Rate and False Negative Rate, we adjusted the threshold.
Different Feature Extractor ?
We tried going down the route of a different feature extractor, in this case, we tried the extractor that our professor had built for his competition. The extractor chose a subset of the attributes available through the PE File Format and then blew it up into a 200,000 feature space. Sadly, all our laptops crashed under this heavy load and we couldn't get the university systems to work for us in the duration of the competition. So we gave up on that idea and went back to the Ember Feature Extractor. For anybody who might be interested and has the resources to run it, here it is.
Finding more data
The next idea was to add on to our database. And instead of blindly trying out different datasets, we decided to target datasets. For this purpose we created a Vocabulary Extractor script, that basically walks through directories of PE files and creates a set (vocabulary) of the strings it sees. This could be the import functions, the exports functions, the datadirectories, entrypoint etc.
Here's a preview of the vocabulary we created out of the test samples
imports->USER32.dll->GetDC
datadirectories->name->CERTIFICATE_TABLE
imports->KERNEL32.dll->CreateSemaphoreA
imports->KERNEL32.dll->IsBadStringPtrA
imports->KERNEL32.dll->WriteFile
imports->KERNEL32.dll->EndUpdateResourceW
imports->KERNEL32.dll->GetProcessPriorityBoost
imports->KERNEL32.dll->WideCharToMultiByte
imports->KERNEL32.dll->EnterCriticalSection
header->optional->subsystem->WINDOWS_GUI
We also created a vocabulary from the ember dataset that our model was training on. Using another script called Vocabulary Comparer, we found the difference between these vocabularies to figure out where our model was lacking. The strategy once we found these "missing" words was to write a script that downloads malware files from the internet, extracts the vocabulary of the file and then compares it with the "missing words" to decide whether to keep the file or not. However, due to lack of time we were not able to build this script and hence had to go back to trial-and-erroring through different dataset.
Avoiding dataset operations on RAM
Here's another issue we came up against when we were trying to train our models. The dataset was getting too big to be contained and processed on in the RAM. More specifically, we had multiple datasets that should be combined into one training set which was then meant to be fed into the model as input. However just combining it all together was eating through my RAM and causing my computer to crash. So we had to figure out a better way. This is where numpy's memmap came into the picture. With this function, we were able to do the combining process on disk without loading it into memory.
Final Model
After a couple of all-nighters or near all-nighters, we came up with a couple of different models that were honestly not performing good enough. We tried a combination of different models in a pipeline approach and finally got to a model that was performing comparatively better than others: Random Forest with 100 estimators(named Zoe) trained on Ember2018 dataset, and a Random Forest with 300 estimators (named Mike from Monsters Inc.) trained on Ember2018, Benign-NET and Bodmas all of which uses the PE format attributes to detect malware. Here's the docker image for our model.
Attack
For the attack part, we parallely started testing out how appending random strings and benign strings would influence the detection, as well as getting a dropper working.
Dropper
So our professor, had built a dropper for this purpose that we decided to base our work on. The dropper takes in the malware file as a payload in its resources and then when executed, writes (drops) the payload into the disk and executes it. The dropper itself was made to emulate a calculator PE file. The beauty of the dropper is that it entirely changes the PE file attributes into the attributes of this enclosing dropper application while the real payload, and all its attributes are enclosed as a resource.
Packers
We also tried out a lot of packers (some were very sketchy). While some of them did increase the evasiveness, it was not enough to get the detection systems to change their decisions. Moreover, the well known packers made the detection systems become more suspicious of our samples, making it a counter-productive strategy.
Steganography?
After spending time getting used to the compilation process for PE Files (we used the MSVC toolset so that we can automate the process easily), and a couple of debugging sessions we got the dropper up and running. With this dropper we came up with a strategy of using steganography to further improve our evasiveness. So while the professor embeds the entire payload as a resource, we planned to hide the payload inside a regular image. Every 1 byte out of 8 bytes used in the picture would be containing the malware data. The plan was to use this image (that contained the payload) as an icon resource for our dropper. Pretty soon, we realized that hiding a payload of 1KB would require an image of 8KB and hence this would break a criteria of the competition (Not more than 5MB increase in size) for most payloads. So, then we decided to forget hiding the malware but just to convert it into an image.
Image Conversion
We were able to get this image strategy working using OpenCV initially. However not in time for the first blind submission of attacks, and hence we submitted the basic version of the professor's dropper that we got working. After this submission, we were given access to the other group's models inorder to perform white-box attacks on them. With this OpenCV, the performance of the adversarial samples were kinda inconsistent. For many models, and for many samples, the strategy worked brilliantly while for others it reduced the evasiveness. An even bigger issue was that when we checked the funcionality of the adversarial samples we realized that we broke functionality for all of the fies. This was because the sample required the presence of OpenCV library in the target system for its functioning. A little research into dynamic linking and static linking made us understand the issue we were facing. For the exe file to work without being dependent on having OpenCV on the target system, we would have to statically link it while compiling, and just reading about it showed that it's not completely straightforward. Instead we started looking for other image manipulation libraries that would make our lives easier and we stumbled upon lodepng. The whole point of lodepng is its portability. It just had one source file and one header file and could easily be linked to our exe without added complexities. It took me a couple of days to get it working, as the data kept being corrupted after the whole binary->image->binary conversion.
So now that we had the image startegy working, it turned out to be not the magic solution that we were expecting it to be. Like I said it before, it was good for some files and some models, but not for all of them. So, time for a different strategy.
The way the dropper emulates a benign files is that it has a lot of dead functions that would make the sample look benign. We decided to target this approach towards our competitor's models. So the strategy is, to find out benign samples that were targeted with a high confidence by a specific model as goodware. Then using the vocabulary extractor script built during the defender stage, get the imports out. Then add these functions to our dropper dead function set. The idea is to try and emulate a PE file that we know for a fact that the model we are targeting considers to be a goodware. With this strategy we, were actually able to get the evasiveness up by a good margin.
In the meanwhile, since we had access to the other teams' detection systems (docker images), we were also able to make them print out the confidence scores with which they base their decisions on. This would help us observe with finer details, how each strategy and each change affects each model. To know how we managed to modify the docker images to get to the model details, check out the blog post by my friend, Sidharth Baveja.
So armed with a couple of different strategies with varying rates of success we analyzed the results. Instead of going for one strategy that works for every situation, we decided to go for choosing the best strategy for a specific malware file, that is, the strategy that beats the most number of models. Using this approach, we further increased our rate of success. Once we reached this level, we started using the appending strings strategy in a random trial-and-error fashion (Hail Mary) to get some samples over the line to flip the decision of the detection system from malware to goodware.
Results
Our team came in first place for the attack phase of the competition!
Link for the project repository here
