Wednesday, March 30, 2016

#Tay Twist: @Tayandyou Twitter Account Was Hijacked ...By Bungling Microsoft Test Engineers (Mar. 30)

[Update 5:35am  From CNBC http://www.cnbc.com/2016/03/30/tay-microsofts-ai-program-is-back-online.html:
Microsoft's artificial intelligence (AI) program, Tay, reappeared on Twitter on Wednesday after being deactivated last week for posting offensive messages. However, the program once again went wrong and Tay's account was set to private after it began repeating the same message over and over to other Twitter users. According to a Microsoft, the account was reactivated by accident during testing.
"Tay remains offline while we make adjustments," a spokesperson for the company told CNBC via email. "As part of testing, she was inadvertently activated on Twitter for a brief period of time." (emphasis added)
I'm puzzled by this explanation but I'll go back through the evidence to see which explanation is best supported.]

[Update 6:35am  It now looks like the "account hack" was really a bungled test session by someone at Microsoft Research -- effectively a "self-hack".

Important: This episode was not "Tay being Tay".]

The @Tayandyou Twitter chatbot has been silent since last Thursday when Microsoft shut it down. Shortly after midnight today, Pacific time, the @Tayandyou Twitter account woke up and started blasting tweets at very high volume.  All of these tweets included other Twitter handles in them, maybe from previous tweets, maybe from followers.

But it became immediately apparent that something was different and wrong.  These tweets didn't look anything like the ones before, in style, structure, or sentience.  From the tweet conversations and from the sequence of events, I believe that the @Tayandyou account was hacked today (March 30), and was active for 15 minutes, sending over 4,200 tweets.

[Update 4:30am
The online media has started posting articles, but they all treat this as more "Tay runs amok".  Only The Verge has updated their story.  If you read an article that doesn't at least consider that Tay's Twitter account was hacked, could you please add a comment with link to this post?  Thanks.]

Here's an example of Hijacked-Tay's tweets (credit to @daviottenheimer):

(click to enlarge)

Notice that @TestAccountInt1 is in all these tweets.  That account is very new, and probably tied to this takeover:

 

Notice the second to last tweet: "straight up drivin the hot mess express CHOO CHOOOOO".  This is a very distinct expression, and if this was the real Tay chatbot, there would most likely be another tweet exactly like this somewhere in Twitter history.  But I could only find two:


[Update 4:20am -- Corrected:]
The first is a retweet of AI-Tay (Mar 23).  The second tweet may or may not be a historical tweet (i.e. pre-Tay), but there is no date.  I tried to find it through search inside Twitter and I couldn't find it.

Hijacked-Tay's most frequent tweet, by far, was "You are too fast, please take a rest...".  Here is a very small sample:


At about 12:15am, I flagged the account as "hacked" on the account page for @Tayandyou.  Just a minute later, @Tayandyou tweets were set to "protected", so I couldn't see them any more.


Other folks (including @rogue_analyst) could still see @Tayandyou, but said that the tweet activity had ceased.  A few minutes after that, no one could see tweets from @Tayandyou.  Either Twitter took down the account or the hackers did themselves, having achieved their laughs.  Here is a sample of how human users reacted:



During this takeover, Hijacked-Tay sent out over 4,200 tweets in about 15 minutes.  Current tweets = "100K", while it was "95.8K" when it was shut down.


Notice that @Tayandyou's location is "the internets".  Notice that the location for @TestAccountInt1, above, is "Da Internet".  This is suggestive evidence that @TestAccountInt1 was created with @Tayandyou in mind.

Let's see if this episode gets any press attention, and how they treat it.  I wonder if they will attribute it to Microsoft and more misbehavior because of AI poisoning?

[Update 2:30am]

Looks like someone is cleaning up the account.  Tweet count was reduced steadily from 100K back down to 95K, which is 800 less than when Tay was shut down last week.  Also all the pictures that were uploaded (mostly just text saying "You are too fast, please take a rest...") were also deleted.

Yes, there has been some press coverage from online media.  No investigation and they don't mention hijack.  But at least they present some information:

These articles treat this restart as if it were the real AI-Tay chatbot, seeing all these tweets as just another example of antisocial behavior that it "learned" last week.  Several articles are just copied/repeated from other sources (a.k.a. "recycled beer foam").

They do include some interesting new tweet content from Hijacked-Tay, including this:

Another very distinct phrase.  "Kush" is slang for marijuana.  The line comes from a rap song by Gucci Mane called "Ordinary Gangsta".  It's not a copy/paste of historical tweets, however.  It's not even copy/paste from the song.   I believe this was typed by a human, one of the hackers who took over the account.

Here is some video of the tweet stream, with sound track added. Not very informative.

[Update 3:30am]

There's a threat on 4chan.com/pol/ here.  Mostly they are just reacting to the new activity.  No evidence that anyone there is involved in the hack.  In fact, none of the posters seem to be aware that it was hacked, and instead they believe this was AI-Tay.

There is some new information, including images posted by Hijacked-Tay.  This was apparently the last image posted by Hijacked-Tay.


The binary decodes into ascii text:


Looks like the work of a hax0r to me, for the lulz.

2 comments:

  1. Hi,
    You asked to be notified about articles considering the possibility of Tay getting hacked. Here's one in Finnish.

    http://www.hs.fi/m/talous/a1459310261562?jako=0262aaf74fa4d0064c58dc460f6e61d7&ref=tw-share

    ReplyDelete