GenAI tools highlight potential flaws in Grant Applications

ChatGPT is continuing to show the cracks in established documentation processes and “gate-keeping” systems…

Historically the written word has been used as part of the application process to judge the quality of the applicant(ion) and to dissuade casual applicants by requiring a certain level of effort.  Along with the enormous problems around GenAI authored student essays (which most academics would consider cheating) it appears that academics are getting on in the act by experimenting with GenAI to write proposals.

Is this cheating or simply using the most modeern tool for the job?   

In a recent article in Nature, the author (J.M. Parilla)  expresses a dislike for writing grant applications due to the extensive amount of work involved: Grant applications, he explains,  often require various documents, such as a case for support, a lay summary, an abstract, mulitple CV’s, impact statements, public engagement plans, project management details, letters of support, data handling plans, risk analysis, and more. Despite this extensive (expensive) effort, there’s a very high chance of rejection (90–95%).

The author suggests that the system is flawed, time-consuming, and cumbersome. The focus during the review process, he argues, is often on whether the proposal ticks a number of boxes in cluding whether it aligns with the call brief (including the format), if the science is good and novel and if the candidates are experts in the field.

The author decided to use ChatGPT as a tool that could assist in writing grant proposals which, he claims,  reduced the workload significantly. The author therefore questions the value of asking scientists to write documents that AI can easily create, suggesting it might be time for funding bodies to reconsider their application processes.

He notes that a recent Nature survey indicates a significant number of researchers (>25%) are already using AI to aid in writing manuscripts and (>15%) asdmit to using it for grant proposals. Whilst the article acknowledges that some may view using ChatGPT as “cheating” it argues that it underscores a larger issue in the current grant application system.

It concludes that the fact that artificial intelligence can do much of the work makes a mockery of the process and argues that it’s time to make it easier for scientists to ask for research funding.

Ai-Da Jubilee portrait fails Turing test

In a unexpected public demonstration of rare AI expertise by the art world,  a recent review by the Guardian art critic opined that a portrait of HM Queen Elizabeth II by AI artist Ai-Da “fails to meet the Turing test” (sic) though we are unclear what version of the Turing test he is referring to since the reviewer was only looking at one piece and already knew the piece in question was done by an AI before correctly guessing that it was done by an AI.

He went on to say:

“This delusion works by deliberately ignoring the huge gap between the current state of machine learning and the dream of true AI, which would pass the Turing test and match the complexity of the human mind. Ai-Da is not an artist because she – or rather it – has no independent consciousness.”

Whilst we are doubtless completely comfortable that the critic in question doesn’t LIKE the piece, it would be interesting to see if he could actually follow a Turing-like protocol and point out the human artist in a blind review of human- vs AI-generated pieces. We think it would be fun to watch – like trying to distinguish pictures of real faces or human text from AI generated ones.

 Its unclear whether the Guardian critic really dislikes Ai-Da’s style or is uncomfortable that last month, she held her first solo exhibition at the 2022 Venice Biennale.

When asked for comment about the Critics knowledge of AI, Ai-Da apparently smiled and said he had failed the Turing Test as he did not give a convincing impression of being intelligent, at least not on this particular topic.

In conversation with: Jennifer Zhu Scott

In conversation this time is well-known finance and digital economy expert, Jennifer Zhu Scott. Jen recently joined the WST Board of Trustees and we are delighted to welcome her. Ian Brown sat down to find out a little more about Jennifer’s (Jen’s) path to Web Science and why she thinks we’ve invented a whole new kind of poverty and what we should be doing about it.

Ian: Hi Jen, welcome to the Trust and thanks for joining us today to give the WST members and supporters an idea of who you are and where your interests lie.

Jen: No problem – I’m really pleased to be joining the board at a time when there is so much important work to do.

Ian: Like many of us you didn’t start out as a Web Scientist but reading your Bio you have studied very widely across different disciplines in Sichuan, Manchester and many top institutions – that’s quite a journey – can you tell us a little about it?

Jen: I was brought up in an environment where my father was always tinkering, disassembling and reassembling radios, fixing lights and telephones. I was very comfortable with technology. When I was in university, I bought the parts and built my own PC. Technology and science is my native language. I remember being fascinated by what technology could do. Today, as a professional, it is evident that technology has transformed every aspect of our life. Whilst our understanding of technology leaped ahead at a breakneck pace, our understanding of the social impacts of technology (the socio-technological aspect)  has been moving much, MUCH slower. I knew there must be trade-offs between what technology could do and what it should do but there didn’t seem to be any good models or guidelines for that. Arguably there still aren’t.  

My studies started with Applied Maths & Computer Science and when I left China I came to the UK to work and later on studied Finance in my master’s degree. Data is the essence of every discipline I’ve studied. I moved into industry working for some big FinTech data companies looking at how advanced technologies could be applied to businesses individually and what the key trends would be in (digital) value.  However, I was still interested in how all these benefits could be distributed across society more broadly and continued my studies branching into public policy – trying to understand how policy is formed and how change is driven on a larger scale.



Ian: You mentioned the importance of data and you gave a TED talk in 2019 about data and why we should be being getting paid for it

Jen: Absolutely. We are supposed to work towards a more inclusive and equitable economy, but in terms of data ownership, most of us are just equally poor. Most people haven’t understood the concept or implications of data poverty. The thing I learned in China as a child is that ownership, personal ownership, brings a form of liberty and the opportunity for improvement. At a time when seven of the top 10 companies on the planet derive their wealth from data about us, the conclusion is that data is immensely valuable – but the power struggle for the ownership and control of the data has only been between corporates and governments, and individuals have no seat at the table, yet the vast majority of data is generated by individuals.  My proposal of establishing the economic value of individuals’ data with a degree of pricing power is a way to grant the individuals’ rights in a digital economy and reflect each individual’s nuanced need for privacy.

Ian: I think it’s widely accepted that when a product is offered for free it is generally the users who are actually the product. I like to think of it as receiving “free shovels” that we use to dig up all the vegetables in our garden and give away to the supermarket where we can go to buy them back! 

Jen: I would argue that in the case of the current economy, we are not even a product. Shoshana Zuboff writes in her book “The Age of Surveillance Capitalism” that we are only raw materials in the current digital economy. I tend to agree with her. We also give away our time, privacy, and mental wellbeing to constantly produce data for big tech.  I argue that in many ways our ‘free will’ is an illusion – a result of algorithms to manipulate more attention and more ad clicks. Therefore, a nuanced reflection of our privacy, health and individual priorities in our digital life is an important pillar of a fair and inclusive digital economy.

Ian: That is a constant problem on the Web – finding models that fit everyone globally.

Jen: In Europe, California, and increasingly China, the regulators approach this problem with more and more limitations and regulations. In China, to respond to centralized sensitive data collection and control, the regulators are introducing data localization rules to protect national security. There are more than 60 regulators around the world that are working on more than 150 various data localization rules. But the web is supposed to transcend borders and jurisdictions. Instead of forcing a balkanization of the World Wide Web, we should enable and empower decentralized data control and ownership that puts the individual at the center. With a decentralised model, it would be harder for one corporate to put national security at risk. 

Ian: We are seeing a lot of debate about Elon Musk’s proposal to change policy at Twitter if/when he buys it. In simplistic terms are we trading free speech against hate speech?

Jen: Twitter has become a tremendously powerful platform with its algorithm driving political and social discussions around the world and whether or not Elon believes he is championing free speech for all the right reasons we have to question whether one person should be making decisions with such a huge potential impact for hundreds of millions of people around the world. Elon is using his position to improve things as he sees them, but ultimately even a “better Emperor” is still an Emperor.

Ian: So you are suggesting more regulation of these types of technologies?

Jen: As we discussed, global regulation may not always be appropriate at the local level – this is where public policy comes in. There is an important difference between asking HOW something is done and if something SHOULD be done. Technology is a bit like medicine – we should be exploring, developing, and investigating what is possible without necessarily automatically licensing/approving every discovery, everywhere before understanding the costs, trade-offs, and local impacts.  This is about value-driven leadership  – moving beyond profits towards benefits and improvements for society as a whole.

Ian: But would you support the large-scale use of personal data in some cases? Some people argue that small amounts of data “don’t count” ..

Jen: Arguing that individual data doesn’t count is like arguing that one vote doesn’t count – it’s the principle that counts and it certainly matters to the individual. Data at scale is valuable of course – the question is who has the control. I chair The Commons Project, a tech non-profit that’s working towards interoperability and global health data standards that will allow us to respond to national and international events like pandemics by quickly sharing data between different countries and labs globally so the borders won’t need to shut down for so long. Covid has shown us the need to be able to react quickly and globally. At The Commons Project, we do not monetize individuals’ data. While there is a large amount of data in the mix, we minimize the data collection and maximize privacy protection. With the right governance model, you can build tech that puts the people at the center.

Ian: So with use cases like this that employ global technical standards for health data where is the place for Web Science?

Jen: Web Science brings together a host of interdisciplinary approaches from technology, law, philosophy, medicine, government (and many more) to examine the issues and decide the most important questions; even if we can do something, when/where is it appropriate to do so? How can we do it so there is clear accountability to the people and society? 

Historical medical data about a terminated pregnancy might inform health policy generally and future medical treatment for that one patient specifically but it might also get that patient prosecuted, imprisoned (or worse) in certain legal jurisdictions, or where policy/public opinion may change over time. We need to think beyond the narrow impact (or profit) in the present and consider the longer-term, wider strategic impact of these decisions.  

Ultimately the question is much more nuanced than “how can we capture/store the data?”.

In China, the ride service DIDI collected detailed journey/location information on over 550 million passengers and 10’s of millions of drivers. DIDIs aggregated data on billions of journeys offered detailed maps/models of locations that were not even on official maps and that showed who had been where and when. When attempting a foreign (US) listing in 2021 the Chinese government became uncomfortable about the international security and privacy implications of the data and has moved to restrict DIDI’s operations through the removal of the associated apps from mobile platforms as well as an investigation of the company’s potential abuses of personal data.

It goes to show that data and networks of data “at scale” have very different social implications to smaller private data stores – Web Science focuses on these types of networks at global scale.

Ian: What do you see as the role of Web Science going forward? What would you like to see happen?

Jen: We should be looking to educate users about how their data is used, how valuable it is, and why they should be managing it better. In Web2, companies like Facebook have data monetization baked into their business model. Their algorithm is designed to hook users to spend more time on their site because ‘time on site’ is an important determinant of advertising pricing. What the algorithm discovered is that when people are angry they tend to stay engaged for the longest time. This is why platforms like Facebook are full of divisive, provocative content that’s designed to trade your rage for advertising dollars. We live in a more and more polarised and divided world so Mark Zuckerberg can become a multibillionaire. Web Science Trust should gather the brightest minds in the world in our field to actively educate, debate, participate and build a healthier digital world.  There are so many more issues to address – how AI interacts with our data, the responsibility for the algorithms, the crypto-asset bubble, the lack of security and value model for NFT and the list goes on. It all centers around data: the use of data. the value of data, the ethics of data, and the ownership of data.

If our view of the world on the Web (what we see and what we are served up via search and social media) remains so strongly controlled by a combination of a data-centric 360-degree profile of our activities and profit-centered algorithms then I would argue that it’s not only a huge privacy issue, as people have argued – our freedom of information, our freedom to choose and, with it, our free-will are severely impacted. Does free will actually become an illusion?   

We need an impactful, multi-disciplinary conversation about data: its value, its uses, its ownership, and its potential benefits for society – that is where Web Science can and must make an impact.

Ian: Jen – thanks for joining us and once again welcome to the Web Science Trust!

Early NFT investor embarrassed by no resale interest

The recent surge in NFT coverage in tyhe technology and financial press was typified by the story of an early NFT auction in which an investor paid $2.9m for an NFT linked to Twitter Founder Jack Dorsey’s first ever Tweet on the platform. This, it was claimed, was an example of how new value could be created using NFTs and how investing in buying and selling NFTs would be the next big thing. The sales of NFTs have indeed grown large though the residual value, and ROI on the resale of NFTs have been much less impressive.

In the interim it has been widely reported not only that NFT exchanges have struggled with growing numbers or fraudulent issues, fake NFTs, market manipulation, price rigging and thefts (ironically specifically the issues that block chain technologies are intended to prevent) but also that a lot of the apparent liquidity in NFT markets (the number of buy/sell transactions) has been artificially (and illegally) inflated by the same parties being on both sides of the transaction in any attempt to give the impression that NFT are easily/quickly traded and that prices are going up.

In what must be considered a massive PR blow to the industry as a whole, the buyer of this famous first Tweet NFT, Sina Estavi has recently tried to sell what is probably the most famous NFT in existence for sale referring to it as the “Mona Lisa of the Digital World” for an eye-watering $48m on the OpenSea NFT exchange (asking more than 16x what he paid for it in March 2021) and was met with initially offers of only hunderds of dollars and at the time of writing a highest bid of just $6’800 – some 0.000141 of the asking price and barely 0.0023 of what he paid for it. Surely a “rug pull” of epic proportions.

Our condolensces go to Mr Estavi who had allegedly planned to donate about $25m of the expected proceeds to charity though we think he has badly mis-judged what he has purchased. While he claims this NFT is the “Mona Lisa of the Digital World” – something which would indeed be priceless, surely what he has actually bought is an NFT for a photo of the Mona Lisa which can be bought in any gift shop for a few dollars or downloaded from the Web for free. 

Perhaps more correctly he bought $2.9m of attention/publicity for himself and the new NFT exchange he is launching whilst the NFT that remains behind when the news stories and buzz are forgotten actually captures very little inherent value beyond the attention and novelty they generate. Who was the second woman to fly solo across the Atlantic? The second athelete to break a 4-minute mile? No-one remembers and I suspect the the second person to own this NFT might be equally forgetable.

Perhaps NFTs are becoming a new currency of the “attention economy” joining subscribers, likes and upvotes. In any case potential investors must now surely be ultra-careful about certificates pointing to notional assets that are hard (impossible) to differentiate from the free alternatives.

Whack-a-Mole at Cent NFT exchange and piracy on the OpenSea

The US-based NFT trading platform, Cent, which stunned an unsuspecting public by selling an NFT of Jack Dorsey’s first ever tweet on Twitter for £2.1 million has recently suspended trading of (most) NFT assets because “people were selling tokens of content that did not belong to them”, with its co-founder Cameron Hejazi admitting this was a “fundamental problem” in the fast-growing digital assets market. Whilst the Cent marketplace has stopped general NFT sales, the part specifically for selling NFTs of tweets, which Cent calls “Valuables” (sic) is still operating.

“There’s a spectrum of activity that is happening that basically shouldn’t be happening – like, legally” said co-founder Hejazi, highlighting three main problems:

  • People selling unauthorized copies of other NFTs (aka Counterfeiting)
  • People making NFTs of content which does not belong to them (aka Fraud)
  • People selling sets of NFTs which resemble a security (aka Securities Fraud)

Additionally there is (at least) a fourth problem – there has been substantial reported activity where the same party buys/sells their own NFTs (so-called wash-trading) in order to make the NFT market or asset appear more valuable/popular than it actually is. In a market where (quite apart from the value of the asset it points to) the token itself has virtually no value other than the public interest in it and so this amounts to serious market manipulation. This is a form of securities fraud which has been illegal in the US since 1936.

Hejazi was apparently not trying to downplay the situation when he described these issues as “rampant,” with users “minting and minting and minting counterfeit digital assets.” “It kept happening. We would ban offending accounts but it was like we’re playing a game of whack-a-mole… Every time we would ban one, another one would come up, or three more would come up.”

It appears that there is potential systemic and market risk in a marketplace which cannot effectively prevent users fraudulently trading on its systems and the liability for any losses incurred might seem to be a troubling future possibility for the operators of such platforms.

Its not difficult to see the appeal of the model: It seems to follow naturally from the notion that the one thing better than making money from things you make/buy/sell is making money from things that OTHER PEOPLE make/buy/sell.  A model delivering huge financial sucess to Amazon MarketPlace, Apple AppStore and eBay in recent years.

After all, NFTs require neither physical storage space nor distribution network, require neglible power and no materials to make, require no maintenance and yet (currently) sell for what can only be described as “incredible” prices (i.e., not believable). If you are an exchange or trading platform for NFTs than you don’t even need to expend any effort on creating the assets in the first place. What modern digital company wouldn’t want a piece of that emerging market? For those old enough to remember buying virtual t-shirts and virtual baseball caps for real money in Second Life – we seem to be heading back to the digital store to buy more digital stuff – one wonders whether this is primarily because our lives and our homes are already full to the brim of the physical stuff  that we’ve been buying for the last decade or two.

How can firms continue to grow and place products in markets that are already choking with stuff and where the cost of manufacturing, materials, offsetting pollution and disposal of waste are becoming such hot topics? The answers to these problems may become more visible as more global brands join the NFT gold rush focussed on selling us pointers/receipts to things we don’t actually own. Amazon solves this problem elegantly as they increasingly sell products manufactured stored shipped by other people and as they move from paper books (assets which we actually own when we buy them and can be sold or given away) to Kindle eBooks which we do NOT own but rather license from Amazon. Whilst licenses vary beteeen platforms and can change over time essentially the rights to read an ebook typically end with the death of the orginal customer. Makes you wonder what will happens with NFTs: will they be “bearer instruments” like a bond that anyone can carry around or named contracts like modern equity registers? Will they be timeless like a book or time-bound like an ebook license? So many questions… 

With Coca-Cola, Gucci and Nike featuring among companies to have already sold NFTs, Nike has even bought a virtual sneaker maker to sell digital shoes  (no mention of where our digital socks or our digital feet will be coming from).  Alphabet-owned company YouTube has said it will explore NFT features, presumably to further license and monetise the huge amounts of digital content on its platform (perhaps as an alternative to ad revenues) or the confusing YouTube Red aka YouTube Premium. 

Confidence issues however are more widespread then just around a single platform.  The biggest NFT marketplace, OpenSea, currently valued at $13.3 billion, said last month more than 80% of the NFTs minted for free on its platform were “plagiarized works, fake collections and spam.” For the avoidance of doubt an OpenSea spokesperson did confirm that “It is against our policy to sell NFTs using plagiarized content”. This will perhaps be of minimal comfort to anyone who has already handed over good money for a fake NFT, has seen valid NFTs of their copyright works illegally minted or more generally to the investors behind OpenSea. One wonders if the connection between (the) OpenSea and the traditional home of pirates occured to the marketing department at the time they were choosing a name or if indeed the universe really is powered by irony as some have suggested.

 More recently “at least three opportunists exploited the OpenSea loophole Monday, making away with over $1 million worth of ether in ensuing NFT sales, according to blockchain analytics firm Elliptic. One user (jpegdegenlove), paid roughly $133,000 for seven NFTs, before flipping the digital collectibles on OpenSea for $934,000 of ether.”  The issue is being characterised as a “UI Issue”. According to Blockworks OpenSea reportedly had a bug in its marketplace that destroyed 42 NFTs last year, but was fixed within a day. 

Despite the fairly obvious conclusion that markets littered with fakes, stolen assets and fraudsters are not only “problematic” (but are perhaps to be avoided entirely), sales of NFTs have nevertheless rocketed to around £18 billion in 2021, leaving many baffled (including the author) as to where the money is coming from and why so much of it is being spent on items that do not physically exist and which typically anyone can view online for free.

Its a very very good question.

To many NFT-enthusiasts, the decentralised nature of blockchain technology is appealing, allowing users to create and trade digital assets without a central authority controlling the activity, though it appears that a lack of a central authority may, ironically, be NFT’s single biggest weakness if it cannot quickly and successfully assert a credible and trustworthy way of creating and trading digital assets. Cent has talked about putting centralised controls in place whilst they figure out how to make the platform work.

Who are they for?

Much has been made of the ability for musicians and artists to recoup some of their diminishing copyright revenues as art and content production has increasingly moved into the world of streaming and downloads – the question, however, remains what can you DO with a token for an asset:

  • That potentially everyone else has access to anyway (without paying)
  • That confers no control nor ownership rights over the original asset

There seem to be a few reasons so far:

  1.  Speculate with it – i.e. attempt to re-sell it while the price is still going up. Otherwise, like a game of pass-the-parcel , the market may eventually determine the asset’s price is largely determine by buzz  (see Wash-Trading) rather than any underlying value and is now worth less than you paid for it. This makes NFTs feel like a quick-in-quick-out trading opportunity with the guy holding the NFT parcel when the music stops potentially badly out of pocket.
  2. Gain attention (participate) with it – i.e. derive a vicarious pleasure and attention from owning something that notionally links you with a famous item or person whether or not the certificate is eventually worth more than you paid for it. There is a story about US actor John de Lancie (who played the iconic Star Trek character known as “Q”) appearing at an auction of sci-fi memorabilia despite feeling very ill with the flu whereupon a fan paid $60 for his unfinished glass of water in order to obtain “a copy of the Q-Virus.” People will, it seems, buy virtually anything if the story behind it is suitably compelling. John de Lancie accepted the $60 on behalf of a charity and has fully recovered whilst little is known about where the virus is now.
  3. Support art/culture with it – i.e. find a method of supporting artists and content producers who have had previously rivalrous goods like albums, CDs and videos replaced by non-rivalrous and heavily-pirated digital versions. NFTs might be a method to support artists though a connection to older works/pieces but whether this is more attractive and represents a better relationship between the artist and the fans than something like a Patreon subscription remains to be seen,

Other than appeasing die-hard Star Trek fans (which is probably not the primary market the NFT exchanges are looking at) the more interesting question for the longevity of NFT is whether you can invest in them – given they don’t age or degrade (unlike the assets they point to). There will never be fewer of them (only more) and like signed books, albums, autographs and celebrity-owned items, the risk is that they are (currently) easy to fake and thus the market is full of counterfeits. Remember even a valid NFT isn’t the real thing- only a certificate. Its like buying the certificate for a Hendrix guitar but not actually the guitar … I’m not sure I see the attraction.

Critics have observed that in a poorly regulated market NFT’s may also be a convenient method to soak up (or even launder) large amounts of cash that individuals and organisations would prefer local tax authorities knew little about. Anti-money laundering groups are doubtless keeping a close eye on this space since a recent US Treasury report on money laundering and art works.

Looking back to the evolution of earlier electronic financial markets 1980’s-1990’s there were undendeniably widespread problems in understanding how the markets worked and how the assets should be priced. NFTs may experience these same problems as markets develop with early wild over-pricing being replaced by more conservative methods and practices. 

Broader block-chain technologies and digital currencies based on proof-of-work or proof-of-control are doubtless interesting approaches with growing applications (albeit some with ugly environmental costs) and NFTs are part of this broader landscape.  Without better regulation and processes the current NFT landscape may be no more than the latest get-rich-quick scheme for traders to get in and out before the markets collapse. With better understanding tools and regulation we might be seeing the formation of new markets in novel digital assets that will actually hold value and fund arts/business for longer term investors to capture and increase value over time.

Time will tell how quickly the NFT Goldrush can be made safe and secure for the general public and the general investor but for now anyone putting up serious cash in an unproven market for an uncertain digital token is living in the wild west.

It is perhaps worth noting that the only people who consistently make money in the wild west during the gold rushes were the businesses selling shovels.


Terms/ Concepts

NFTs (or Non-Fungible Tokens) are crypto assets that record the ownership of a digital asset such as an image, video or text. Anyone can create, or “mint,” an NFT, however ownership of the token does not usually confer control over or ownership of the underlying item nor is there (as yet) a reliable method of ensuring that the individual minting the NFT has any right to do so especially where they do not own the original underlying asset.

Fungible: Despite having serial numbers Notes/Bills (incl pound notes and dollar bills!), stock certificates, bond certificates and similar physical financial certificates are handled de facto as though they are identical and can be freely swapped for another bill, note or certificate of the same denomination – they are fungible-  i.e. equivalent/interchangeable. NFT’s however represent unique and NON-Fungilble records which are interesting because of their unique nature.

Rights: When we paint a watercolour scene on paper and sell it we no longer have the painting after the sale: the purchaser does. These are “rivalrous goods” such that EITHER I have the painting or the buyer does. Digital goods in contrast may be “non-rivalrous” in that if we paint a digital portrait and sell the image we can still have what is effectively the SAME image as the purchaser after the sale – differences between two copies of an unsigned digital image file can range from tiny to non-existent. Digital rights, especially around non-rivalrous goods like digital assets may be poorly defined/understood in many areas leading to misunderstandings around what rights/materials the buyer is obtaining.

Scams: Despite the intention to digitally certifiy individual ownership reliably, reports of scams, counterfeits and so-called “wash trading” have become commonplace. Individuals are minting reproductions of supposedly unique NFTs pointing to underlying assets, minting NFTs pointing to assets (including artworks) over which they have no rights.

Wash trading: “wash trading” is where the same individual is on both sides of the trade (buy and sell) which attempts to paint a misleading picture inflating the apparent interest in an asset and thereby inflating the perception of the asset’s value and liquidity.

Government agencies are tapping a facial recognition company to prove you’re you – here’s why that raises concerns about privacy, accuracy and fairness


 Beginning this summer, you might need to upload a selfie and a photo ID to a private company,, if you want to file your taxes online.

Oscar Wong/Moment via Getty Images

James Hendler, Rensselaer Polytechnic Institute

The U.S. Internal Revenue Service is planning to require citizens to create accounts with a private facial recognition company in order to file taxes online. The IRS is joining a growing number of federal and state agencies that have contracted with to authenticate the identities of people accessing services.

The IRS’s move is aimed at cutting down on identity theft, a crime that affects millions of Americans. The IRS, in particular, has reported a number of tax filings from people claiming to be others, and fraud in many of the programs that were administered as part of the American Relief Plan has been a major concern to the government.

The IRS decision has prompted a backlash, in part over concerns about requiring citizens to use facial recognition technology and in part over difficulties some people have had in using the system, particularly with some state agencies that provide unemployment benefits. The reaction has prompted the IRS to revisit its decision.

a webpage with the IRS logo in the top left corner and buttons for creating or logging into an account




Here’s what greets you when you click the link to sign into your IRS account. If current plans remain in place, the blue button will go away in the summer of 2022.
Screenshot, IRS sign-in webpage

As a computer science researcher and the chair of the Global Technology Policy Council of the Association for Computing Machinery, I have been involved in exploring some of the issues with government use of facial recognition technology, both its use and its potential flaws. There have been a great number of concerns raised over the general use of this technology in policing and other government functions, often focused on whether the accuracy of these algorithms can have discriminatory affects. In the case of, there are other issues involved as well.

ID dot who? is a private company that formed as TroopSwap, a site that offered retail discounts to members of the armed forces. As part of that effort, the company created an ID service so that military staff who qualified for discounts at various companies could prove they were, indeed, service members. In 2013, the company renamed itself and started to market its ID service more broadly. The U.S. Department of Veterans Affairs began using the technology in 2016, the company’s first government use.

To use, a user loads a mobile phone app and takes a selfie – a photo of their own face. then compares that image to various IDs that it obtains either through open records or through information that applicants provide through the app. If it finds a match, it creates an account and uses image recognition for ID. If it cannot perform a match, users can contact a “trusted referee” and have a video call to fix the problem.

A number of companies and states have been using for several years. News reports have documented problems people have had with failing to authenticate them, and with the company’s customer support in resolving those problems. Also, the system’s technology requirements could widen the digital divide, making it harder for many of the people who need government services the most to access them.

But much of the concern about the IRS and other federal agencies using revolves around its use of facial recognition technology and collection of biometric data.

Accuracy and bias

To start with, there are a number of general concerns about the accuracy of facial recognition technologies and whether there are discriminatory biases in their accuracy. These have led the Association for Computing Machinery, among other organizations, to call for a moratorium on government use of facial recognition technology.

A study of commercial and academic facial recognition algorithms by the National Institute of Standards and Technology found that U.S. facial-matching algorithms generally have higher false positive rates for Asian and Black faces than for white faces, although recent results have improved. claims that there is no racial bias in its face-matching verification process.

There are many other conditions that can also cause inaccuracy – physical changes caused by illness or an accident, hair loss due to chemotherapy, color change due to aging, gender conversions and others. How any company, including, handles such situations is unclear, and this is one issue that has raised concerns. Imagine having a disfiguring accident and not being able to log into your medical insurance company’s website because of damage to your face.




Facial recognition technology is spreading fast. Is the technology – and society – ready?

Data privacy

There are other issues that go beyond the question of just how well the algorithm works. As part of its process, collects a very large amount of personal information. It has a very long and difficult-to-read privacy policy, but essentially while doesn’t share most of the personal information, it does share various information about internet use and website visits with other partners. The nature of these exchanges is not immediately apparent.

So one question that arises is what level of information the company shares with the government, and whether the information can be used in tracking U.S. citizens between regulated boundaries that apply to government agencies. Privacy advocates on both the left and right have long opposed any form of a mandatory uniform government identification card. Does handing off the identification to a private company allow the government to essentially achieve this through subterfuge? It’s not difficult to imagine that some states – and maybe eventually the federal government – could insist on an identification from or one of its competitors to access government services, get medical coverage and even to vote.

As Joy Buolamwini, an MIT AI researcher and founder of the Algorithmic Justice League, argued, beyond accuracy and bias issues is the question of the right not to use biometric technology. “Government pressure on citizens to share their biometric data with the government affects all of us — no matter your race, gender, or political affiliations,” she wrote.

Too many unknowns for comfort

Another issue is who audits for the security of its applications? While no one is accusing of bad practices, security researchers are worried about how the company may protect the incredible level of personal information it will end up with. Imagine a security breach that released the IRS information for millions of taxpayers. In the fast-changing world of cybersecurity, with threats ranging from individual hacking to international criminal activities, experts would like assurance that a company provided with so much personal information is using state-of-the-art security and keeping it up to date.

[Over 140,000 readers rely on The Conversation’s newsletters to understand the world. Sign up today.]

Much of the questioning of the IRS decision comes because these are early days for government use of private companies to provide biometric security, and some of the details are still not fully explained. Even if you grant that the IRS use of the technology is appropriately limited, this is potentially the start of what could quickly snowball to many government agencies using commercial facial recognition companies to get around regulations that were put in place specifically to rein in government powers.

The U.S. stands at the edge of a slippery slope, and while that doesn’t mean facial recognition technology shouldn’t be used at all, I believe it does mean that the government should put a lot more care and due diligence into exploring the terrain ahead before taking those critical first steps.The Conversation

James Hendler, Professor of Computer, Web and Cognitive Sciences, Rensselaer Polytechnic Institute

This article is republished from The Conversation under a Creative Commons license. Read the original article.

James Hendler, Professor of Computer, Web and Cognitive Sciences, Rensselaer Polytechnic Institute
This article is republished from The Conversation under a Creative Commons license.