Fished the Twitter API implementation, everything works as of writing :)

This commit is contained in:
robinuniverse 2021-07-12 13:49:19 -05:00
parent 8aa811e5f6
commit 2a638ac4b4
3 changed files with 50 additions and 24 deletions

View File

@ -1,3 +1,3 @@
{
"test": "test"
}
}

View File

@ -1,8 +1,6 @@
# TwitFix
very basic flask server that fixes twitter embeds in discord by using youtube-dl to grab the direct link to the MP4 file and embeds the link to it in a custom page, with link caching
This does work! but I'm new to flask, so it can probably be improved a great deal.
Flask server that serves fixed twitter video embeds to discord by using either the Twitter API or Youtube-DL to grab tweet video information
## How to use (discord side)
@ -10,7 +8,9 @@ just put the url to the server, and directly after, the full URL to the tweet yo
**I now have a copy of this running on a Linode server, you can use it via the following url**
https://fxtwitter.com/`twitter video url` or `last half of twitter url (everything past twitter.com/)`
```
https://fxtwitter.com/[twitter video url] or [last half of twitter url] (everything past twitter.com/)
```
You can also simply type out 'fx' directly before 'twitter.com' in any valid twitter video url, and that will convert it into a working TwitFix url, for example:
@ -20,9 +20,7 @@ You can also simply type out 'fx' directly before 'twitter.com' in any valid twi
## How to run (server side)
this script uses the youtube-dl python module, along with flask and pymongo, so install those with pip (you can use `pip install -r requirements.txt`) and start the server with `python twitfix.py` ( will need sudo if you leave it at port 80 )
By default I have the port set to 80, just cause that's what was convenient for me, but it can easily be changed, either using an environment variable, or changing the bottom line of the script itself
this script uses the youtube-dl python module, along with flask, twitter and pymongo, so install those with pip (you can use `pip install -r requirements.txt`) and start the server with `python twitfix.py`
I have included some files to give you a head start on setting this server up with uWSGI, though if you decide to use uWSGI I suggest you set up mongoDB link caching
@ -39,12 +37,12 @@ TwitFix generates a config.json in its root directory the first time you run it,
- **db**: Caches all links to a mongoDB database. This should be used it you are using uWSGI and are not just running the script on its own as one worker
- **json**: This saves cached links to a local **links.json** file
**method** - ( Options: **youtube-dl**, **api**, **hybrid** ) **[NOT YET IMPLEMENTED!]**
**method** - ( Options: **youtube-dl**, **api**, **hybrid** )
- **youtube-dl**: the original method for grabbing twitter video links, this uses a guest token provided via youtube-dl and should work well for individual instances, but may not scale up to a very large amount of usage
- **api**: this directly uses the twitter API to grab tweet info, limited to 900 calls per 15m
- **hybrid**: This will start off by using the twitter API to grab tweet info, but if the rate limit is reached it will switch over to youtube-dl to avoid downtime
- **hybrid**: This will start off by using the twitter API to grab tweet info, but if the rate limit is reached or the api fails for any other reason it will switch over to youtube-dl to avoid downtime
This project is licensed under the **Do What The Fuck You Want Public License**

View File

@ -6,7 +6,6 @@ import json
import re
import os
app = Flask(__name__)
pathregex = re.compile("\\w{1,15}\\/status\\/\\d{19}")
@ -101,20 +100,49 @@ def vidInfo(url, tweet="", desc="", thumb="", uploader=""): # Return a dict of v
}
return vnf
def linkToVNF(vidlink):
try:
print("Attempting to download tweet info from Twitter API")
twid = int(re.sub(r'\?.*$','',vidlink.rsplit("/", 1)[-1])) #gets the tweet ID as a int from the passed url
tweet = twitter_api.statuses.show(_id=twid)
url = tweet['extended_entities']['media'][0]['video_info']['variants'][-1]
vnf = vidInfo(url, vidlink, tweet['text'], tweet['media']['media_url'], tweet['user']['name'])
def linkToVNFfromAPI(vidlink):
print("Attempting to download tweet info from Twitter API")
twid = int(re.sub(r'\?.*$','',vidlink.rsplit("/", 1)[-1])) # gets the tweet ID as a int from the passed url
tweet = twitter_api.statuses.show(_id=twid, tweet_mode="extended")
if tweet['extended_entities']['media'][0]['video_info']['variants'][-1]['content_type'] == "video/mp4":
url = tweet['extended_entities']['media'][0]['video_info']['variants'][-1]['url']
else:
url = tweet['extended_entities']['media'][0]['video_info']['variants'][-2]['url']
vnf = vidInfo(url, vidlink, tweet['full_text'], tweet['extended_entities']['media'][0]['media_url'], tweet['user']['name'])
return vnf
def linkToVNFfromYoutubeDL(vidlink):
print("Attempting to download tweet info via YoutubeDL")
with youtube_dl.YoutubeDL({'outtmpl': '%(id)s.%(ext)s'}) as ydl:
result = ydl.extract_info(vidlink, download=False)
vnf = vidInfo(result['url'], vidlink, result['description'], result['thumbnail'], result['uploader'])
return vnf
except Exception:
print("API Failed, Attempting to download tweet info via YoutubeDL")
with youtube_dl.YoutubeDL({'outtmpl': '%(id)s.%(ext)s'}) as ydl:
result = ydl.extract_info(vidlink, download=False)
vnf = vidInfo(result['url'], vidlink, result['description'], result['thumbnail'], result['uploader'])
return vnf
def linkToVNF(vidlink): # Return a VideoInfo object or die trying
if config['config']['method'] == 'hybrid':
try:
return linkToVNFfromAPI(vidlink)
except Exception:
print("API Failed")
return linkToVNFfromYoutubeDL(vidlink)
elif config['config']['method'] == 'api':
try:
return linkToVNFfromAPI(vidlink)
except Exception as e:
print("API Failed")
print(e)
return None
elif config['config']['method'] == 'youtube-dl':
try:
return linkToVNFfromYoutubeDL(vidlink)
except Exception as e:
print("Youtube-DL Failed")
print(e)
return None
else:
print("Please set the method key in your config file to 'api' 'youtube-dl' or 'hybrid'")
return None
def getVNFfromLinkCache(vidlink):
if link_cache_system == "db":