Gather all Comments from a Project

Overview

The comment scraper will retrieve all Assets in a Project, scrape the Comments, and output them in a .csv file. This walkthrough uses the following resources:

Python SDK - The SDK handles pagination for you, and sets up a client you can use with developer tokens (bearer authentication).
Comment scraper in Python - The code sample we are using is here.

Try it on Glitch

Glitch is a simple tool allows you to set up your own applications and test and run them using their servers. If you’re not familiar with Glitch, check out our guide on Using Glitch.

If you want to see the comment scraper work on a server, we have a Flask app set up on Glitch: Frame.io Comment Scraper in Python.

Required scopes

Before beginning this guide, you’ll need to make sure you have a token that includes at least the following scopes:

Scope	Reason
Projects: Read	Not strictly required, but necessary for fetching the `root_asset_id` of a Project.
Assets: Read	Necessary for navigating through `asset_ids` via API
Comments: Read	Necessary for retrieving the Comments themselves.

1. Prepare your app

This guide will follow a similar pattern as Reading the File Tree — accordingly, to get started, you’ll need:

The root_asset_id of the Project you’re crawling
Your Developer Token

You’ll also want to import the FrameioClient into your Python app, as well as some additional helper libraries:

Python

1 from frameioclient import FrameioClient
2 import requests, json, csv, itertools
3 
4 ROOT_ASSET_ID = "<ROOT_ASSET_ID>"
5 TOKEN = "<DEV_TOKEN>"

2. Crawl the Project

Now, you’ll need to recursively retrieve all Assets, check them for Comments, and stash any Commented Assets in list from which you can construct your .csv file.

As you crawl through your Project, you’ll need to make the following checks on each Asset:

Is the Asset a file? ("_type": "file").
- If it’s a file, does it have Comments?
  - If it does, fetch them and add to your list.
Is the asset a folder? ("_type": "folder")
- If it’s a folder, then recurse on its children.
Is the asset a Version Stack? ("_type": "version_stack")
- If it’s a Version Stack, then get all the children and check for Comments.

Python

1 def all_comments(client, asset_id, comment_list):
2     files = client.get_asset_children(asset_id)
3 
4     for asset in files:
5         if asset['type'] == "file":
6             if asset['comment_count'] > 0:
7                 asset_parent_id = asset['parent_id']
8                 asset_name = asset['name']
9                 comments = client.get_comments(asset['id'])
10                 my_comment_list = [comment for comment in comments.results]
11                 for object in my_comment_list:
12                     object.update({'parent_id':asset_parent_id})
13                     object.update({'name':asset_name})
14                 comment_list.append(my_comment_list)
15 
16         if asset['type'] == "folder":
17             if asset['item_count'] > 0:
18                 all_comments(client, asset['id'], comment_list)
19 
20         if asset['type'] == "version_stack":
21             asset_name = asset['name']
22             parent_id = asset['parent_id']
23             vfiles = client.get_asset_children(asset['id'])
24 
25             for asset in vfiles.results:
26                 asset_name = asset['name']
27                 parent_id = asset['parent_id']
28                 if asset['type'] == "file":
29                     if asset['comment_count'] > 0:
30                         comments = client.get_comments(asset['id'])
31                         my_comment_list = [comment for comment in comments.results]
32                         for object in my_comment_list:
33                             object.update({'parent_id':parent_id})
34                             object.update({'name':asset_name})
35                         comment_list.append(my_comment_list)
36 
37 def get_all_project_comments(root_asset_id, token):
38     comment_list = []
39     client = FrameioClient(token)
40 
41     all_comments(client, root_asset_id, comment_list)
42 
43     return comment_list

Don't forget to paginate

The example above ignores pagination — but as you navigate through large collections, you shouldn’t! See our Key Concepts guide for reference.

3. Flatten your Comments list

Unless you handled this during your crawl, you’ll want to flatten out your list for easier processing.

Python

1 # The response list comes back as a list of lists.
2 # Flatten out responses to a single list so you can build your .csv file.
3 
4 flat_response_list = list(itertools.chain.from_iterable(responses))

4. Cull the list and create your .csv

When your list is flat, you can use list comprehension to grab the elements from each comment that you think are useful, and then output them into a .csv file. We recommend at least having:

Comment - text
Parent ID - parent_id
Asset ID - asset_id
Asset Name name
Owner ID - owner_id
Owner Email - owner.email
Timestamp - timestamp
Updated At - updated_at

When you’ve made a new list containing the output you want per asset for your final output, it’s time to write to your .csv file.

Python

1 list_for_csv = [[o['text'], o['parent_id'], o['asset_id'], o['name'], o['owner_id'], o['owner']['email'], o['timestamp'], o['updated_at']] for o in flat_response_list]
2 
3 # Let's write our new list out to a .csv file. We'll add a heading.
4 with open("output.csv", 'w') as myfile:
5      wr = csv.writer(myfile, dialect='excel')
6      wr.writerow(['Comment', 'Parent ID', 'Asset ID', 'Asset Name', 'Owner ID', 'Email', 'Timestamp', 'Updated At'])
7      wr.writerows(list_for_csv)

And that’s it! You now have a .csv file with the flattened Comments from an entire Frame.io Project.

1	from frameioclient import FrameioClient
2	import requests, json, csv, itertools
3
4	ROOT_ASSET_ID = "<ROOT_ASSET_ID>"
5	TOKEN = "<DEV_TOKEN>"

1	def all_comments(client, asset_id, comment_list):
2	files = client.get_asset_children(asset_id)
3
4	for asset in files:
5	if asset['type'] == "file":
6	if asset['comment_count'] > 0:
7	asset_parent_id = asset['parent_id']
8	asset_name = asset['name']
9	comments = client.get_comments(asset['id'])
10	my_comment_list = [comment for comment in comments.results]
11	for object in my_comment_list:
12	object.update({'parent_id':asset_parent_id})
13	object.update({'name':asset_name})
14	comment_list.append(my_comment_list)
15
16	if asset['type'] == "folder":
17	if asset['item_count'] > 0:
18	all_comments(client, asset['id'], comment_list)
19
20	if asset['type'] == "version_stack":
21	asset_name = asset['name']
22	parent_id = asset['parent_id']
23	vfiles = client.get_asset_children(asset['id'])
24
25	for asset in vfiles.results:
26	asset_name = asset['name']
27	parent_id = asset['parent_id']
28	if asset['type'] == "file":
29	if asset['comment_count'] > 0:
30	comments = client.get_comments(asset['id'])
31	my_comment_list = [comment for comment in comments.results]
32	for object in my_comment_list:
33	object.update({'parent_id':parent_id})
34	object.update({'name':asset_name})
35	comment_list.append(my_comment_list)
36
37	def get_all_project_comments(root_asset_id, token):
38	comment_list = []
39	client = FrameioClient(token)
40
41	all_comments(client, root_asset_id, comment_list)
42
43	return comment_list

1	# The response list comes back as a list of lists.
2	# Flatten out responses to a single list so you can build your .csv file.
3
4	flat_response_list = list(itertools.chain.from_iterable(responses))

1	list_for_csv = [[o['text'], o['parent_id'], o['asset_id'], o['name'], o['owner_id'], o['owner']['email'], o['timestamp'], o['updated_at']] for o in flat_response_list]
2
3	# Let's write our new list out to a .csv file. We'll add a heading.
4	with open("output.csv", 'w') as myfile:
5	wr = csv.writer(myfile, dialect='excel')
6	wr.writerow(['Comment', 'Parent ID', 'Asset ID', 'Asset Name', 'Owner ID', 'Email', 'Timestamp', 'Updated At'])
7	wr.writerows(list_for_csv)