Does it have to be a government? Why not a third party non-profit? The white hat gets shielded, and the non-profit has credible lawyers which makes suing them harder than individuals.
The idea is to make it easier to fix the vulnerability than to sue to shut people up.
For credit assignment, the person could direct people to the non profit’s website which would confirm discovery by CVE without exposing too many details that would allow the company to come after the individual.
This business of going to the company directly and hoping they don’t sue you is bananas in my opinion.
1) If you make legal disclosure too hard, the only way you will find out is via criminals.
2) If other industries worked like this, you could sue an architect who discovered a flaw in a skyscraper. The difference is that knowledge of a bad foundation doesn’t inherently make a building more likely to collapse, while knowledge of a cyber vulnerability is an inherent risk.
3) Random audits by passers-by is way too haphazard. If a website can require my real PII, I should be able to require that PII is secure. I’m not sure what the full list of industries would be, but insurance companies should be categorically required to have an cyber audit, and laws those same laws should protect white hats from lawyers and allow class actions from all users. That would change the incentives so that the most basic vulnerabilities are gone, and software engineers become more economical than lawyers.
In other industries there are professional engineers. People who have a legal accountability. I wonder if the CS world will move that way, especially with AI. Since those engineers are the ones who sign things off.
For people unfamiliar, most engineers aren't professional engineers. There are more legal standards for your average engineer and they are legally obligated to push back against management when they think there's danger or ethics violations, but that's a high bar and very few ever get in legal trouble, only the most egregious cases. But professional engineers are the ones who check all the plans and the inspections. They're more like a supervisor. Someone who can look at the whole picture. And they get paid a lot more for their work but they're also essential to making sure things are safe. They also end up having a lot of power/authority, though at the cost of liability. Think like how in the military a doctor can overrule all others (I'm sure you've seen this in a movie). Your average military doctor or nurse can't do that but the senior ones can, though it's rare and very circumstantial.
You'd be surprised how many SE's would love for this to happen. The biggest reason, as you said, being able to push back.
Having worked in low-level embedded systems that could be considered "system critical", it's a horrible feeling knowing what's in that code and having no actual recourse other than quitting (which I have done on few occasions because I did not want to be tied to that disaster waiting to happen).
I actually started a legal framework and got some basic bills together (mostly wording) and presented this to many of my colleagues, all agreed it was needed and loved it, and a few lawyers said the bill/framework was sound .. even had some carve-outs for "mom-n-pops" and some other "obvious" things (like allowing for a transition into it).
Why didn't I push it through? 2 reasons:
1.) I'd likely be blackballed (if not outright killed) because "the powers that be" (e.g. large corp's in software) would absolutely -hate- this ... having actual accountability AND having to pay higher wages.
2.) Doing what I wanted would require federal intervention, and the climate has not been ripe for new regulations, let alone governing bodies, in well over a decade.
Hell, I even tried to get my PE in Software, but right as I was going to start the process, the PE for Software was removed from my state (and isn't likely to ever come back).
I 100% agree we should have even a PE for Software, but it's not likely to happen any time soon because Software without accountability and regulation makes WAY too much money ... :(
> You'd be surprised how many SE's would love for this to happen
I'm one of them, and for exactly the reason you say.
I worked as a physical engineer previously and I think the existence of PEs changes the nature of the game. I felt much more empowered to "talk back" to my boss and question them. It was natural to do that and even encouraged. If something is wrong everyone wants to know. It is worth disruption and even dealing with naive young engineers than it is to harm someone. It is also worth doing because it makes those engineers learn faster and it makes the products improve faster (insights can come from anywhere).
Part of the reason I don't associate my name with my account is so that I can talk more freely. I absolutely love software (and yes, even AI, despite what some might think given my comments) but I do really dislike how much deception there is in our industry. I do think it is on us as employees to steer the ship. If we don't think about what we're building and the consequences of them then our ship is beholden to the tides, not us. It is up to us to make the world a better place. It is up to us to make sure that our ship is headed towards utopia rather than dystopia (even if both are more of an idea than reality). I'd argue that if it were up to the tides then we'll end up crashing into the rocks. It's much easier to avoid that if we're managing the ship routinely than in a panic when we're headed in that direction. I think software has the capacity to make the world a far better place. That we can both do good and make money at the same time. But I also think the system naturally will disempower us. When we fight against the tides things are naturally harder and may even look like we're moving slower. But I think we often confuse speed and velocity, frankly, because direction is difficult to understand or predict. Still, it is best that we try our best and not just abdicate those decisions. The world is complex, so when things work they are in an unstable equilibrium. Which means small perturbations knock us off. Like one ship getting stuck shutting down a global economy. So it takes a million people and a billion tiny actions to make things go right and stay right (easier to stay than fix). But many of the problems we hate and are frustrated by are more stable states. Things like how wealth pools up, gathered by only a few. How power does the same. And so on. Obviously my feelings extend beyond software engineering, but my belief is that if we want the world to be a better place it takes all of us. The more that are willing to do something, the easier it gets. I'd also argue that most people don't need to do anything that difficult. The benefit and detriment of a complex machine is that small actions have larger consequences. Just because you're a small cog doesn't mean you have no power. You don't need to be a big cog to change the world, although you're unlikely to get recognition.
I think you’re taking the professional responsibility that engineers are given too far. They are not given that responsibility to make political decisions, as you seem to be implying. Engineers are professionals in the hard sciences, not in social sciences. They only have power over ethical and safety issues directly pertaining to technical matters. I think ethics in this sense includes only very widely accepted ethical opinions, not anything that people from different political parties would disagree on. Engineering, in other words, is not political. Making the world better, as you put it, is something that requires political decisions. I hope people don’t make this confusion because the last thing most of us would like to see is Engineering becoming a political endeavor, including software engineering.
I also come from a more "traditional engineering" background, with PEs and a heavier sense of responsibility/ethics(?). I definitely think that's where it's going, although in my somewhat biased opinion, that's why the bar for traditional engineering in terms of students and expected skill and intuition was much higher than with CS/CE, which means the get rich quick scheme nature of it might go away.
In many countries you are only allowed to call yourself a Software Engineer if you actually have a professional title.
It is countries like US where anyone can call themselves whatever they feel like that have devalued our profession.
I have been on the liability side ever since, people don't keep broken cars unless they cannot afford anything else, software is nothing special, other than lack of accountability.
I don’t think the current cost structure of software development would support a professional engineer signing their name on releases or the required skill level of the others to enable such …
We’d actually have to respect software development as an important task and not a cost to be minimized and outsourced.
We check the output of engineers tjats what infra audits and certs are for. We basically tell industry if you want to waste your money on poor engineers whose output doesn’t certify go ahead.
you could do that with civil engineering. anyone gets to design bridges. bridge is done we inspect, sorry x isn’t redundant your engineering is bad tear it down.
You couldn't do that with civil engineering, because checking if a bridge was built correctly is actually really hard, and it's why it's such a process for engineers to sign off on phases of construction.
A lot of responses below talking about what a 'certified' or 'chartered' engineer should be able to do.
I thought it would be noteworthy to talk about another industry, accountancy. This is how it works in the UK, but it is similar in other countries. They are called 'Chartered Accountants' here, because their institute has a Royal Charter saying they are the good guys.
To become a Chartered Accountant has no prerequisites. You 'just' have to complete the qualification of the institute you want to join. There are stages to the exams that prior qualifications may gain you exemptions from. You also have to log practical experience proving you are working as an accountant with adequate supervision. It takes about 2-3 years to get the qualification for someone well supported by their employer and with sufficient free time. Interestingly many Accountants are not graduates, and instead took technician level qualifications first, often the Association of Accounting Technicians (AAT). The accounting graduates I have interviewed wasted 3 years of their lives...
There are several institutes that specialise in different areas. Some specialise in audit. One specialises in Management Accounting (being an accountant at a company really). The Management accountants one specifically prohibits you from doing audit without taking another conversion course. All the institutes have CPD requirements (and check) and all prohibit you from working in areas that you are not competent, but provide routes to competency.
There are standards to follow, Generally Accepted Accounting Practice GAAP, UK Financial Reporting Standards FRS and the International equivalent IFRS. These cover how Financial Statements are prepared. There are superate standards setting bodies for these. There are also a set of standards that cover how an audit must be done. Then there is tax law. You are expected to know them for any area you are working in. All of these are legally binding on various types of corporation. See how that switches things around? Accountants are now there to help the company navigate the legal codes. The directors sign the accounts and are liable for misstatements, that encourages them to have a director who is an accountant...an audit committee etc.
How does that translate to software?
There are lots of standards, NIST, GDPR, PCI, some of which are legally or contractually binding. But how do I as a business owner know that a software engineer is competent to follow them. Maybe I am a diving company that wants a website. How do I know this person or company is competent to build it? It requires software engineers with specific qualifications that say they can do it, and software engineers willing to say, 'I'm sorry I am not able to work in this field, unless I first study it'.
I’m big on increasing accountability and responsibility for software engineering, but I’ve learned about SEI CMMI, and worked in an ISO 9001 shop.
In some cases, these types of structures make sense, but in most others, they are way overkill.
It’s a conundrum. One of the reasons for the crazy growth of software, is the extreme flexibility and velocity of development, so slamming the brakes on that, would have enormous financial consequences in the industry (so … good luck with that …).
But that flexibility and velocity is also a big reason for the jurassic-scale disasters that are a regular feature of our profession. It’s entirely possible for people that are completely unqualified, to develop software full of holes. If they can put enough lipstick on it, it can become quite popular, with undesirable consequences.
I don’t think that the answer is some structured standard and testing regime, but I would love to see improvement.
Regarding your 2), in other industries and engineering professions, the architect (or civil engineer, or electrical engineer) who signed off carries insurance, and often is licensed by the state.
I absolutely do not want to gatekeep beginners from being able to publish their work on the open internet, but I often wonder if we should require some sort of certification and insurance for large businesses sites that handle personal info or money. There'd be a Certified Professional Software Engineer that has to sign off on it, and thus maybe has the clout to push back on being forced to implement whatever dumb idea an MBA has to drive engagement or short-term sales.
Maybe. Its not like its worked very well lately for Boeing or Volkswagen.
> I absolutely do not want to gatekeep beginners from being able to publish their work on the open internet
FWIW there is no barrier like that for your physical engineers. Even though, as you note, professional engineers exist. Most engineers aren't professional engineers though, and that's why the barrier doesn't exist. We can probably follow a similar framing. I mean it is already more common for licensing to be attached to even random software and that's not true for the engineer's equivalents.
Oh there have been many cases where software engineers who are not professional engineers with the engineering mafia designation get sidelined by authorities for lacking standing. We absolutely should get rid of the engineering mafias and unions.
It's kinda wild that you don't need to be a professional engineer to store PII. The GDPR and other frameworks for PII usually do have a minimum size (in # of users) before they apply, which would help hobbyists. The same could apply for the licensure requirement.
But also maybe hobbyists don't have any business storing PII at scale just like they have no business building public bridges or commercial aircraft.
Web is already mostly centralized, and corporations which should be scrutinized in way they handle security, PII and overall software issues are without oversight.
It is also a matter of respect towards professionals. If civil engineer says that something is illegal/dangerous/unfeasible their word is taken into the account and not dismissed - unlike in, broadly speaking, IT.
I just don't feel we want the overhead on software. I'm in an industry with PEs and I have beef with the way it works for physical things.
PII isn't nearly as big a deal as a life tbh. I'd rather not gatekeep PII handling behind degrees. I want more accoubtability, but PEs for software seems like it's ill-suited for the problem. Principally, software is ever evolving and distributed. A building or bridge is mostly done.
I, as a self-proclaimed dictator of my empire, require, in the name of national security, all chat applications developed or deployed in my empire to send copies of all chat messages to the National Archive for backup in a form encrypted to the well-known National Archive public key. I appoint Professional Software Engineers to inspect and certify apps to actually do that. Distribution of non-certified applications to the public or other forms of their deployment is prohibited and is punishable by jail time, as well as issuing a false certification.
Sounds familiar?
The difference from civil engineering is that governments do not (yet?) require a remotely triggerable bomb to be planted under every bridge, which would, arguably, help in a war, while they are very close to this in software. They do something similar routinely with manufacturing equipment - mandatory self-disabling upon detecting (via GPS) operation in countries under sanctions.
GDPR doesn't have any minimum size before applying. There's a household exemption for personal use, but if you have one external user, you're regulated.
Another missing link is here is the stock price relationship to security vulnerability history of the corporation. Somehow, I don't know how, but somehow stock prices should reflect the corporation's social responsibility posture, part of which is information security obviously.
> If other industries worked like this, you could sue an architect who discovered a flaw in a skyscraper
To match this metaphor to TFA, the architect has to break in to someone else's apartment to prove there's a flaw. IANAL but I'm not positive that "I'm an architect and I noticed a crack in my apartment, so I immediately broke in to the apartments of three neighbours to see if they also had cracks" would be much of a defence against a trespass/B&E charge.
> companies should be categorically required to have an cyber audit
I work with a firm that has an annual pen test as part of its SOC2/GDPR/HIPAA audit, and it's basically an exercise in checking boxes. The pen test firm runs a standard TLS test suite, and a standard web vulnerability test suite, and then they click buttons for a while...
The pen test has never found any meaningful vulnerabilities, and several times drive-by white hats have found issues immediately after the pen test concluded
There are jurisdictions (and cultures) where truth is not an absolute defence against defamation. In other words, it's one thing to disclose the issue to the authorities, it's another to go to the press and trumpet it on the internet. The nail that sticks out gets hammered down.
Given that this is Malta in particular, the author probably wants to avoid going there for a bit. It's a country full of organized crime and corruption where people like him would end up with convenient accidents.
> it's one thing to disclose the issue to the authorities
That's not how any of this works. You are basically arguing for the right to hide criminal actions. Filing with the CSIRT is the only legal action for the white hat to take. This is explicitly by design. Complaining about it is like complaining the police arrested you for a crime you committed.
> it's one thing to disclose the issue to the authorities, it's another to go to the press and trumpet it on the internet.
At least in the US there is a path of escalation. Usually if you have first contacted those who have authority over you then you're fine. There's exceptions in both directions; where you aren't fine or where you can skip that step. Government work is different. For example Snowden probably doesn't get whistleblower protection because he didn't first leak to Congress. It's arguable though but also IANAL
There’s a ton of crossover between your method and RL. I guess instead of directly training on episodes and updating model weights, you just store episodes in RAM and sample from the most promising ones. It could be a neat way of getting out of infamous RL cold start by getting some examples of rewards. Thanks for sharing.
Thanks! You're right that there's a resemblance to RL. The original approach was proposed by Antithesis, and in Part 1 we map it more directly to a mutation-based Genetic Algorithm: stored paths are the population, the x-position scoring is the fitness function, and bit-flip input generation is the mutation operator. There's no recombination and no learned policy but just evolutionary selection pressure on input sequences.
Interesting point about the RL cold start, one could definitely use the paths discovered first through the evolutionary exploration to seed an RL agent's initial experience which could help skip the early random flailing phase.
The key difference from RL is the goal. We're not trying to learn an optimal policy for playing the game and instead we're trying to explore as much of the state space as possible to find bugs. In Part 2 we plug in a behavior model that validates correctness at every frame during exploration (velocity constraints, causal movement checks, collision invariants). The combination is where it gets interesting: autonomous exploration discovers the states, and the behavior model catches when the game violates its own rules. For testing, the main reason we even care about completing each level is that a completed path serves as the base for more extensive exploration at every point along it. If the exploration can't reach the end, by definition we miss a large part of the state space.
Developing, no, but once companies start releasing vehicles onto our shared public streets I have a lot less tolerance for launching science experiments that end up killing bystanders.
I can understand the argument that in the abstract over-regulation kills innovation but at the same time in the US the pendulum has swung so far in the other direction that it’s time for a correction.
I have no tolerance for bystanders being killed in general. If the science experiments kill on average less bystanders I'm all for them, if they don't they should be stopped until made safer.
In this case the judgement is so extreme because the judge had no tolerance for Tesla lying in relation to the server logs' existence and what they contained (namely that is was indeed their autopilot that was in full control, had been in full control for almost half an hour, and was not worried at all/not issuing warnings, at the time of the crash)
Why does the fact that there isn’t enough funding for the PhDs that exist imply we should produce fewer of them? At least from what the article mentions, figuring out new and better ways to fight diseases seems like one of the most important problems a human could be working on. In my mind the solution is to provide funding and fix the funding process, not produce fewer scientists.
Also, those scientists already exist. If the US decides not to fund them, they will go produce patents and grow the economies of other places. Many countries wish they could attract the talent that the US does.
<< Why does the fact that there isn’t enough funding for the PhDs that exist imply we should produce fewer of them?
In most of the world, most humans have to move within the realm of available resources. One could easily say that if a manager of US sees too many PhDs, it is natural to conclude that since there is not enough resources to go around, adding more resource consumers is silly. We can argue all over whether it is a good policy, or whether the allocation makes sense, or whether the resources are really not there, but, how is is this a difficult logic gate?
The need for things exists independent of the standalone economic viability of those things. That is the entire point of public funding of various resources, including scientific funding. The “available” resources is a political decision.
Further, reduction in funds for public resources or increase in misery for scientists are not in and of themselves evidence that those resources were over-funded or too cushy. For the research discussed in the article it is quite clearly a political decision, not directly grounded in a need for less medical research.
<< The “available” resources is a political decision.
It invariably always is.
<< The need for things exists independent of the standalone economic viability of those things.
Sure, but there is only so long that can go on funding studying of rather pointless stuff[1] ( added UK example to not be accused of hating on anything in particular US-wise ).
<< Further, reduction in funds for public resources or increase in misery for scientists are not in and of themselves evidence that those resources were over-funded or too cushy.
I am not suggesting that. I am literally saying: there is only so much money. That is it. And if push comes to shove, studies of whether chicken finds humans pretty take a back seat to more pressing matters.
There is a (perhaps apocryphal) story of Michael Faraday showing his new invention of an electric motor to a politician in 1821. He had invented it after investigating strange twitching of a magnetic compass needle.
After seeing the motor, the politician asked “what good is it?” and based on what I can find Faraday either said “what use is a newborn baby” or “one day you’ll be able to tax it”.
So two points: One, you don’t always know things will have a high ROI from the start. Sometimes you just have to be curious. And two, politicians care about the next election in two/four years, not planting trees that won’t bear fruit for 30 years.
We have vast amounts of resources. More than enough to supply the basic needs of everyone in the country.
The US is currently choosing to divert absolutely staggering amounts of those resources away from things we have traditionally valued—science, art, infrastructure, taking care of the least fortunate among us, etc—and using them instead to enrich the already-wealthy, in the most blatant and cruel ways.
There is no possible way this can be spun as being about "available resources". The grift is utterly, 100% transparent.
<< There is no possible way this can be spun as being about "available resources". The grift is utterly, 100% transparent.
Eh, I mean if you put it that way, I suppose all those budgets are just a show and not at all an indication of how utterly fucked we are as a country unless we both:
a) massively reduce spending
b) massively raise taxes
In very real terms, there is only so much money. Some additional money can be borrowed, but we a slowly ( but surely ) reaching a breaking point on that as well.
The issue is: no one is willing to sacrifice anything. And I am sympathetic, but if hard choices are not made now, they will be kinda made for us anyway.
We need to claw back billions and billions and billions of dollars from people for whom it will make zero difference in their daily lives, so that we can spend it on people for whom $100 can change their month, and $10000 can change their life.
Lol. No. We have to massively raise taxes JUST to keep this country afloat financially. The poor people are still fucked. I know it is exactly massively popular to say, which is why you don't see major proponents sans rando online like me.
Hardly, my advice is real, would have a long term positive impact, while, admittedly, inflicting a lot of pain in the process. If there is any benefit to it, it would be that at least the pain would be shared equally across the board allowing for some form of 'misery loves company'.
On the other hand, your advice, at best, is happy clappy populist advice that will, temporarily make some people happy, but will not change the trajectory of the country resulting in the exact same spot only few years from implementation; and that is assuming it can be done in a way that is not immediately subverted..
If anything, I am giving you a real good reason for not just being a cynic, but being a cynic, who can make a change that lasts.
75% of Singaporeans are ethnically Chinese so based on what you are saying it would be worth comparing SG Chinese to Chinese CN on regret since China has a much less robust safety net.
People who score well on probability numeracy are likely better educated and better paid and have more in automatic savings plans. So if someone is maxing out their 401k they don’t feel they need to save more.
The article shows that in the US there is a 25 point gap between high and low income on savings regret, and a 14 point gap between high and low numeracy scores.
In Singapore where savings are more automatic numeracy is a more powerful predictor.
The idea is to make it easier to fix the vulnerability than to sue to shut people up.
For credit assignment, the person could direct people to the non profit’s website which would confirm discovery by CVE without exposing too many details that would allow the company to come after the individual.
This business of going to the company directly and hoping they don’t sue you is bananas in my opinion.
reply