How to Scrape Reddit with Google Scripts

Reddit offers a fairly extensive API that any developer can use to easily pull data from subreddits. You can fetch posts, user comments, image thumbnails, votes and most other attributes that are attached to a post on Reddit.

The only downside with the Reddit API is that it will not provide any historical data and your requests are capped to the 1000 most recent posts published on a subreddit. So, for instance, if your project requires you to scrape all mentions of your brand ever made on Reddit, the official API will be of little help.

You have tools like wget that can quickly download entire websites for offline use but they are mostly useless for scraping Reddit data since the site doesn’t use page numbers and content of pages is constantly changing. A post can be listed on the first page of a subreddit but it could be pushed to the third page the next second as other posts are voted to the top.

Reddit Data in Google Sheets

Download Reddit Data with Google Scripts

While there exist quite a Node.js and Python libraries for scraping Reddit, they are too complicated to implement for the non-techie crowd. Fortunately, there’s always Google Apps Script to the rescue.

Here’s Google script that will help you download all the user posts from any subreddit on Reddit to a Google Sheet. And because we are using pushshift.io instead of the official Reddit API, we are no longer capped to the first 1000 posts. It will download everything that’s every posted on a subreddit.

  1. To get started, open the Google Sheet and make a copy in your Google Drive.
  2. Go to Tools -> Script editor to open the Google Script that will fetch all the data from the specified subreddit. Go to line 55 and change technology to the name of the subreddit that you wish to scrape.
  3. While you are in the script editor, choose Run -> scrapeReddit.

Authorize the script and within a minute or two, all the Reddit posts will be added to your Google Sheet.

Technical Details - How to the Script Works

The first step is to ensure that the script not hitting any rate limits of the PushShift service.

const isRateLimited = () => {
  const response = UrlFetchApp.fetch('https://api.pushshift.io/meta');
  const { server_ratelimit_per_minute: limit } = JSON.parse(response);
  return limit < 1;
};

Next, we specify the subreddit name and run our script to fetch posts in batches of 1000 each. Once a batch is complete, we write the data to a Google Sheet.

const getAPIEndpoint_ = (subreddit, before = '') => {
  const fields = ['title', 'created_utc', 'url', 'thumbnail', 'full_link'];
  const size = 1000;
  const base = 'https://api.pushshift.io/reddit/search/submission';
  const params = { subreddit, size, fields: fields.join(',') };
  if (before) params.before = before;
  const query = Object.keys(params)
    .map(key => `${key}=${params[key]}`)
    .join('&');
  return `${base}?${query}`;
};

const scrapeReddit = (subreddit = 'technology') => {
  let before = '';
  do {
    const apiUrl = getAPIEndpoint_(subreddit, before);
    const response = UrlFetchApp.fetch(apiUrl);
    const { data } = JSON.parse(response);
    const { length } = data;
    before = length > 0 ? String(data[length - 1].created_utc) : '';
    if (length > 0) {
      writeDataToSheets_(data);
    }
  } while (before !== '' && !isRateLimited());
};

The default response from Push Shift service contains a lot of fields, we are thus using the fields parameter to only request the relevant data like post title, post link, date created and so on.

If the response contains a thumbnail image, we convert that into a Google Sheets function so you can preview the image inside the sheet itself. The same is done for URLs.

const getThumbnailLink_ = url => {
  if (!/^http/.test(url)) return '';
  return `=IMAGE("${url}")`;
};

const getHyperlink_ = (url, text) => {
  if (!/^http/.test(url)) return '';
  return `=HYPERLINK("${url}", "${text}")`;
};

Bonus Tip: Every search page and subreddit on Reddit can be converted into JSON format using a simple URL hack. Just append .json to the Reddit URL and you have a JSON response.

For instance, if the URL is https://www.reddit.com/r/todayIlearned, the same page can be accessed in JSON format using the URL https://www.reddit.com/r/todayIlearned.json.

This works for search results as well. The search page for https://www.reddit.com/search/?q=india can be downloaded as JSON using https://www.reddit.com/search.json?q=india.

The 101 Most Useful Websites on the Internet

Here are the most useful websites on the Internet that will make you smarter, increase productivity and help you learn new skills. These incredibly useful websites solve at least one problem really well. And they all have cool URLs that are easy to memorize thus saving you a trip to Google.

101 Useful websites

The Most Useful Websites and Web Apps

  1. archive.is — take a snapshot of any web page and it will exist forever even if the original page is gone.
  2. autodraw.com — create freehand doodles and watch them magically transform into beautiful drawings powered by machine learning.
  3. fast.com — check the current speed of your Internet connection.
  4. slides.com — create pixel-perfect slide decks and broadcast your presentations to an audience of any size from anywhere.
  5. screenshot.guru — take high-resolution screenshots of web pages on mobile and desktops.
  6. dictation.io – accurate and quick voice recognition in your browser itself.
  7. reverse.photos — upload an image and find similar pictures on the web.
  8. copychar.cc – copy special characters and emojis that aren’t on your keyboard.
  9. codeacademy.com – the best place to learn coding online.
  10. noisli.com — ambient noises to help you improve focus and boost productivity.
  11. iconfinder.com – millions of icons for all kinds of projects. Also try icons8.com and flaticon.com.
  12. jotti.org – scan any suspicious file or email attachment for viruses.
  13. wolframalpha.com – gets answers directly without searching   – see more wolfram tips.
  14. earth.google.com – explore beautiful cities, landmarks and orbit the world in 3D from the comfort of your browser.
  15. unsplash.com – the best place to download images absolutely free.
  16. videos.pexels.com — an online library of free HD videos you can use everywhere. Also see videvo.net.
  17. invideo.io — make impressive videos and branded stories for Instagram, Facebook, and YouTube. Also see animoto.com and biteable.com.
Also see: The Best Android Apps
  1. everytimezone.com – a less confusing view of the world time zones.
  2. e.ggtimer.com – a simple online timer for your daily needs.
  3. random.org – pick random numbers, flip coins, and more.
  4. remove.bg — remove the background from any photograph without firing up Photoshop.
  5. myfonts.com/WhatTheFont – upload an image of any text and quickly determine the font family.
  6. fonts.google.com – the best collection of open source fonts that you can use anywhere without restrictions.
  7. fontstruct.com — draw and build your own fonts and use them in any application.
  8. calligraphr.com — transform your handwriting into a real font.
  9. regex.info – find data hidden in your photographs – see more EXIF tools.
  10. youtube.com/webcam — broadcast yourself live over the Internet without any complicated setup.
  11. remotedesktop.google.com — access other computers or allow others to remote access your computer over the Internet.
  12. homestyler.com – design from scratch or re-model your home in 3D.
  13. pdfescape.com – lets you quickly edit PDF in the browser without Acrobat.
  14. draw.io – create diagrams, wireframe and flowcharts in the browser.
  15. web.skype.com — make voice and video calls in your browser with Skype.
  16. onlineocr.net – recognize text from scanned PDFs – see other OCR tools.
  17. wetransfer.com – for sharing really big files online.
  18. file.pizza — peer to peer file transfer over WebRTC without any middleman.
  19. snapdrop.com — like Apple AirDrop but for the web. Share files directly between devices in the same network without having to upload them to any server first.
  20. hundredzeros.com – the site lets you download free Kindle books.
  21. app.grammarly.com — check your writing for spelling, style, and grammatical errors.
  22. noteflight.com – print music sheets, write your own music online ( review).
  23. translate.google.com – translate web pages, PDFs and Office documents.
  24. kleki.com – create paintings and sketches with a wide variety of brushes.
  25. similarsites.com – discover new sites that are similar to what you like already.
  26. bubbl.us – create mind-maps, brainstorm ideas in the browser.
  27. color.adobe.com – get color ideas, also extract colors from photographs.
  28. canva.com — make beautiful graphics, presentations, resumes and more with readymade template designs.
  29. lmgtfy.com – when your friends are too lazy to use Google on their own.
  30. midomi.com – when you need to find the name of a song.
  31. history.google.com —  see all your past Google searches, also among most important Google URLs
  32. faxzero.com – send an online fax for free – see more fax services.
  33. tinychat.com – setup your own private chat room in micro-seconds.
  34. privnote.com – create text notes that will self-destruct after being read.
  35. domains.google.com – quickly search domain names for your next big idea!
  36. squoosh.app – compress images on the fly. Site works offline as well.
  37. downforeveryoneorjustme.com – find if your favorite website is offline or not?
  38. gtmetrix.com – the perfect tool for measuring your site performance online.
  39. builtwith.com — find the web hosting company, email provider and everything else about a website.
  40. urbandictionary.com – find definitions of slangs and informal words.
Also see: The Best Mac Apps and Utilities
  1. seatguru.com – consult this site before choosing a seat for your next flight.
  2. webmakerapp.com — an offline playground for building web projects in HTML, CSS and JavaScript. Also see: glitch.com.
  3. flightstats.com – Track flight status at airports worldwide.
  4. mymaps.google.com – create custom Google Maps with scribbles, pins and custom shapes.
  5. snopes.com – find if that email offer you received is real or just another scam.
  6. typingweb.com – master touch-typing with these practice sessions.
  7. todo.microsoft.com — a beautiful todo app and task manager. Also see Trello.
  8. minutes.io – quickly capture effective notes during meetings.
  9. talltweets.com — Turn Google Slides in animated GIF presentations.
  10. ifttt.com – create a connection between all your online accounts.
  11. namechk.com — search for your desired username across hundreds of social networks and domain names.
  12. gist.github.com — create anonymous and secret text notes and much more.
  13. flipanim.com — create flipbook animations, includes an onion skin tool to let you see the previous frame as you draw the next one.
  14. powtoon.com — create engaging whiteboard videos and presentations with your own voiceovers. Also see videoscribe.co.
  15. clyp.it — Record your own voice or upload an audio file without creating any account. Also see soundcloud.com.
  16. carrd.co — build one-page fully responsive websites that look good on every screen.
  17. spark.adobe.com — make stunning video presentations with voice narration and wow everyone.
  18. anchor.fm — the easiest way to record a podcast that you can distribute on iTunes without have to pay for hosting.
  19. duolingo.com — learn to speak Chinese, French, Spanish or any other language of your choice.
See: The Most Useful Tools for Programmers
  1. buffer.com — the easy way to post and schedule updates on Twitter, Instagram, LinkedIn, Google+ and Facebook.
  2. 10minutemail.com — create disposable email addresses for putting inside sign-up forms.
  3. pixton.com — create your own comic strips with your own characters and move them into any pose.
  4. gravit.io — a full-featured vector drawing tool that works everywhere.
  5. vectr.com — create vector graphics and export them as SVG or PNG files.
  6. twitterbots — create your own Twitter bots that can auto-reply, DM, follow people and more.
  7. headspace.com —  learn the art of meditation and reduct stress, focus more and even sleep better.
  8. forms.studio —  receive files from anyone in your Google Drive with File Upload Forms.
  9. class-central.com — a directory of free online courses offered by universities worldwide.
  10. googleartproject.com — discover museums, famous paintings and art treasure from all around the world.
  11. instructables.com — step-by-step guides on how to build anything and everything.
  12. flowgram.com — make data-driven graphics, charts and infographics. Also see adioma.com and eas.ly.
  13. marvelapp.com — create interactive wireframes and product mockups.
  14. slide.ly — make marketing videos and branded stories for Instagram, Facebook, and YouTube trailers. Also see animoto.com and biteable.com.
  15. photos.icons8.com - make your own stock photographs in high-resolution with custom backgrounds, models and facial expressions.
  16. gohighbrow.com — Take bite-sized courses on a variety of topics, chapters are delivered by email every morning.
  17. htmlmail.pro - send rich-text emails with gmail mail merge.
  18. wirecutter.com — whether you need a vacuum cleaner or an SD card, this is the best product recommendation website on the Internet.
  19. camelcamelcamel.com — Create Amazon price watches and get email alerts when the prices drop.
  20. mockaroo.com — download mock data to fill the rows in your Excel spreadsheet.
  21. asciiflow.com — a WYSIWYG editor to draw ASCII diagrams that you can embed in emails and tweets.
Also see: The Best Add-ons for Gmail, Docs and Sheets
  1. whereami — find the postal address of your current location on Google maps.
  2. sway.com — create and share interactive reports, newsletters, presentations, and for storytelling.
Also see: The Best Websites to Learn Coding
  1. apify.com — the perfect web scraping tool that lets you extract data from nearly any website.
  2. thunkable.com — build your own apps for Android and iOS by dragging blocks instead of writing code.
  3. zerodollarmovies.com — a huge collection of free movies curated from YouTube.
  4. upwork.com — find freelancers and subject experts to work on any kind of project.
  5. duckduckgo.com - a clean alternative to google search that doesn’t track you on the Internet.

Know any useful website that is missing in the list? Please let me know via @twitter.

How to Generate a Report of Bounced Email Addresses in Gmail

Bounced Emails in Gmail

Some email messages you have sent through your Gmail account may not get delivered at all. There could be a problem with the recipient’s email address, like a typo, their mailbox could be full or maybe the mail server could be specifically blocking your emails due to the content of the message.

When an email message sent via Gmail is bounced or rejected, you get an automated bounce-back notice from mailer-daemon@gmail.com and it will always contain the exact reason for the delivery failure along with the SMTP error code. For instance, an error code 550 indicates that the email address doesn’t exist while a 554 indicates that your email was classified as spam by the recipient’s mail server.

How to Get a List of Email Addresses that Bounced

It is important to keep track of your bounced messages and remove all undelivered email addresses from your future mailings as they may affect your sending reputation.

Mail Merge for Gmail keeps track of all your bounced messages in Gmail but if you are not using mail merge yet, here’s an open-source Google Script that will prepare a list of all email addresses that have bounced inside a Google Spreadsheet.

Gmail Bounce Report in Google Sheets

Gmail Bounce Report - Getting Started

Here’s how you can get started:

  1. Click here to make a copy of the Google Spreadsheet.
  2. Open the Bounced Emails menu in your Google Sheet and then select the Run Report option.
  3. Authorize the Google Script so it can scan your Gmail account for bounced emails and write them to the Google Sheet. The script runs entirely in your Google account, no data is stored or shared anywhere.
  4. Watch as the Google Sheet is populated with rejected and bounced email addresses.

The email bounce report includes the email address that bounced, the reason why that email failed to deliver and the date when the bounce occurred. The spreadsheet will also have a direct link to the bounced message received from mailer-daemon.

Technical Details - How the Script Works

The script uses the Gmail API to fetch a list of all bounced emails in your mailbox.

const findBouncedEmails = () => {
  const { messages = [] } = Gmail.Users.Messages.list('me', {
    q: 'from:mailer-daemon',
    maxResults: 200
  });
  for (let m = 0; m < messages.length; m += 1) {
    const bounceData = parseGmailMessage(messages[m].id);
    if (bounceData) {
      SpreadsheetApp.getActiveSheet().appendRow(bounceData);
    }
  }
};

Next, the script parses the headers of bounced email messages with regex and writes the bounced information to the Google Sheet.

const parseGmailMessage = messageId => {
  const message = GmailApp.getMessageById(messageId);
  const body = message.getPlainBody();
  const [, failAction] = body.match(/^Action:\s*(.+)/m) || [];

  /* If failAction is "delayed", igore message since Gmail will retry it */
  if (failAction === 'failed') {
    /* The X-Failed-Recipients header in Gmail contains the recipient's address */
    const emailAddress = message.getHeader('X-Failed-Recipients');
    /* Get the SMTP error code
       The first sub-field indicates whether the delivery attempt was successful
       (2= success, 4 = persistent temporary failure, 5 = permanent failure). */
    const [, errorStatus] = body.match(/^Status:\s*([.\d]+)/m) || [];
    /* The Diagnostic-Code DSN field contains the actual diagnostic code
       Some mail systems supply no additional information beyond that
       which is returned in the 'action' and 'status' fields. */
    const [, , bounceReason] =
      body.match(/^Diagnostic-Code:\s*(.+)\s*;\s*(.+)/m) || [];
    return [
      message.getDate(),
      emailAddress,
      errorStatus,
      bounceReason.replace(/\s*(Please|Learn|See).+$/, ''),
      `=HYPERLINK("${message.getThread().getPermalink()}";"View")`
    ];
  }
  return false;
};

Inside your Google Sheet, go to the Tools menu and choose Script Editor to view the full source code of the Google Script. You are welcome to reuse / modify the code.

Also see: Automatically Unsubscribe from Email Newsletters

How to Force Reset GSuite Users’ Passwords with Apps Script

You can use Google Apps Script to automatically reset the password of users in your GSuite domain. This script can only be executed under the Suite admin account. You also need to enable the AdminDirectory Advanced Service in your Apps Script Editor.

You can force reset passwords of members of a particular group in your organization or specify a list of email addresses and the Google Script will use the AdminDirectory service to change the password of specified users.

const getGroupMembers_ = groupEmail => {
  var emails = [];
  var pageToken;
  do {
    const { members = [], nextPageToken } = AdminDirectory.Members.list(
      groupEmail,
      {
        maxResults: 200,
        pageToken: pageToken
      }
    );
    members.forEach(member => {
      if (member.status === "ACTIVE") {
        emails.push(member.email);
      }
    });
    pageToken = nextPageToken;
  } while (pageToken);
  return emails;
};

const sendEmail_ = (emailAddress, password) => {
  MailApp.sendEmail({
    to: emailAddress,
    cc: "amit@labnol.org",
    subject: `Password changed for ${emailAddress}`,
    body: `The GSuite admin has changed your Gmail password to ${password}`
  });
};

const changePassword_ = emailAddress => {
  const temporaryPassword = Utilities.getUuid();
  AdminDirectory.Users.update(
    {
      password: temporaryPassword,
      changePasswordAtNextLogin: true
    },
    emailAddress
  );
  sendEmail_(emailAddress, temporaryPassword);
};

const resetUserPasswordsForGroup = () => {
  const groupEmail = “groupemail@labnol.org”;
  const members = getGroupMembers_(groupEmail);
  members.forEach(member => changePassword_(member));
};

const resetGSuitePasswordForUsers = () => {
  const members = [
    "user1@example.com",
    "user2@example.com",
    "user3@example.com"
  ];
  members.forEach(member => changePassword_(member));
};

Change GSuite Passwords Periodically

You can create a time-based trigger in Google Scripts to automatically run the reset function at specific intervals (like update password on the first of every month).

ScriptApp
  .newTrigger("resetGSuitePasswordForUsers")
  .timeBased()
  .onMonthDay(1)
  .create();

The Google Script is written in ES6 with V8 runtime. If V8 is not enabled for your GSuite account, replace the manifest appsscript.json file with this:

{
  "timeZone": "Asia/Kolkata",
  "dependencies": {
    "enabledAdvancedServices": [{
      "userSymbol": "AdminDirectory",
      "serviceId": "admin",
      "version": "directory_v1"
    }]
  },
  "exceptionLogging": "STACKDRIVER",
  "runtimeVersion": "V8"
}

Enable Admin Directory Service

To use the Advanced Directory advanced Google service inside your Google Apps Script project, follow these instructions:

  • Open the Google Script, select the Resources menu and then choose Advanced Google services.
  • In the Advanced Google Service dialog that appears, toggle on/off switch next to the Admin Directory service
  • Click OK to save your changes.

How to Install Let’s Encrypt SSL Certificate with Apache on Ubuntu

This step-by-step tutorial will show you how to install Let’s Encrypt SSL certificate for an Apache server running on Ubuntu 18.04. I’ve created a droplet on DigitalOcean for this example but the steps should be similar AWS and other environments.

Install Apache 2

Login to your droplet with root (or use sudo with all the following commands).

Check if any Ubuntu packages are outdated.

apt update

Upgrade the outdated packages to the latest version.

apt upgrade

Install Apache2

apt install apache2

Start the Apache Server

systemctl start apache2

Check if the Apache server is running

systemctl status apache2

Enable the mod_rewrite package for Apache

sudo a2enmod rewrite

Restart Apache

systemctl restart apache2

Install PHP

Install PHP and restart the Apache server.

apt install php libapache2-mod-php
systemctl restart apache2
php —version

Install the CURL package

Install Curl and restart Apache server

apt install curl
apt install php7.2-curl
systemctl restart apache2

Install Let’s Encrypt on Apache

Install the certbot client that will help us automatically manage (install, renew or revoke) the SSL certificates on the Apache server.

Install Certbot

Install the certbot client and the plugin.

sudo apt update
sudo apt-get install software-properties-common
sudo add-apt-repository universe
sudo add-apt-repository ppa:certbot/certbot
sudo apt-get update
sudo apt-get install certbot python-certbot-apache

Install Certbot DNS Plugin

Install the certbot DNS plugin for DigitalOcean. This will automatically add the _acme-challenge TXT DNS records to your domain that are required for authentication. The records are also removed after the certificates are installed.

sudo apt-get install python3-certbot-dns-digitalocean

This will only work if you are using the DigitalOcean Name Servers with your domain.

Create DigitalOcean Credentials File

Go to your DigitalOcean account’s dashboard, choose API and choose “Generate New Token”. Copy the token to your clipboard. Inside the terminal, create a new directory ~/.ssh and create a new file to save the credentials.

vi ~/.ssh/digitalocean.ini

Paste the following line in the credentials file. Replace 1234 with your actual token value.

dns_digitalocean_token = 1234

Save the file and then run chmod to restrict access to the file.

chmod 600 ~/.ssh/digitalocean.ini

Install SSL Certificates

Replace labnol.org with your domain name. This command will install the wildcard SSL certificate for all subdomains and the main domain.

certbot certonly --dns-digitalocean
    --dns-digitalocean-credentials ~/.ssh/digitalocean.ini
    --dns-digitalocean-propagation-seconds 60
    -d "*.labnol.org" -d labnol.org

If the certificate is successfully installed, it will add the certificate and chain in the following directory

/etc/letsencrypt/live/labnol.org/

Test the SSL Certificate

Go to ssllabs.com to test if your new SSL certificate is correctly installed on your domain.

Configure Apache to Use SSL Certificate

Now that the SSL Certificate is installed, we need to enable SSL for the Apache server on Ubuntu.

Enable the SSL module for Apache

OpenSSL is installed with Ubuntu but it is disabled by default. Enable the SSL module and restart Apache to apply the changes.

sudo a2enmod ssl
sudo service apache2 restart

Update Apache Configuration File

Open the default virtual host configuration file /etc/apache2/sites-enabled/000-default.confand paste the following lines. Replace labnol with your domain name.

<VirtualHost *:80>
	RewriteEngine On
	RewriteRule ^(.*)$ https://%{HTTP_HOST}$1 [R=301,L]
</VirtualHost>
<VirtualHost _default_:443>
	ServerAdmin amit@labnol.org
	ServerName labnol.org
	DocumentRoot /var/www/html

	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined

	SSLEngine on
	SSLCertificateFile /etc/letsencrypt/live/labnol.org/fullchain.pem
	SSLCertificateKeyFile /etc/letsencrypt/live/labnol.org/privkey.pem
</VirtualHost>

Save the file and restart Apache. The SSLCertificateFile and SSLCertificateKeyFile files were saved by certbot in the /etc/letsencrypt/live directory.

Adjust the Firewall

In some cases, you may have to enable Apache on SSL port 443 manually with the following command.

sudo ufw allow "Apache Secure"

Restart Apache. All your HTTP traffic will automatically redirect to the HTTPS version with a 301 permanent redirect.

sudo service apache2 restart

Verify Auto-Renewal Process

Your Let’s Encrypt SSL certificate will auto-expire every 90 days. Go to the /etc/cron.d/ folder and you should see a certbot file. This cron job will automatically renew your SSL certificate if the expiration is within 30 days.

You can also run the following command to verify if the renewal process is correctly setup.

sudo certbot renew --dry-run

A Better Way to Embed PDF Documents in Web Pages

How do you embed a PDF document into your website for inline viewing? One popular option is that you upload the PDF file to an online storage service, something like Google Drive or Microsoft’s OneDrive, make the file public and then copy-paste the IFRAME code provided by these services to quickly embed the document in any website.

Here’s a sample PDF embed code for Google Drive that works across all browsers.

  <iframe frameborder="0" scrolling="no"
     width="640" height="480"
     src="https://drive.google.com/file/d/<<FILE_ID>>/preview">
  </iframe>

This is the most common method for embedding PDFs - it is simple, it just works but the downside is that you have no control over how the PDF files are presented in your web pages.

If you prefer to offer a more customized and immersive reading experience for PDFs in your website, check out the new Adobe View SDK. This is part of the Adobe Document Cloud platform but doesn’t cost a penny.

Embed Adobe PDF

Here are some unique features that make this PDF embed solution stand out:

  • You can add annotation tools inside the PDF viewer. Anyone can annotate the embedded PDF and download the modified file.
  • If you have embedded a lengthy document with multiple pages, readers can use the thumbnail view to quickly jump to any page.
  • The PDF viewer can be customized to hide options for downloading and printing PDF files.
  • There’s built-in analytics so you know how many people saw your PDF file and how they interacted with the document.
  • And my favorite feature of ViewSDK is the inline embed mode. Let me explain that in detail.

Display PDF Pages Inline like Images and Videos

In Inline Mode, and this is unique to Adobe View SDK, all pages of the embedded PDF document are displayed at once so your site visitors do not have to scroll another document with the parent web page. The PDF controls are hidden from the user and the PDF pages blend with images and other HTML content on your web page.

To learn more, check this live demo - here the PDF document contains 7 pages but all are displayed at once like one long web page thus offering smooth navigation.

How to Embed PDFs with the Adobe View SDK

It does take a few extra steps to use the View SDK. Go to adobe.io and create a new set of credentials for your website. Please note that credentials are valid for one domain only so if you have multiple websites, you’d need a different set of credentials for them.

Next, open the playground and generate the embed code. You need to replace the clientId with your set of credentials. The url in the sample code should point to the location of your PDF file.

<div id="adobe-dc-view" style="width: 800px;"></div>
<script src="https://documentcloud.adobe.com/view-sdk/main.js"></script>
<script type="text/javascript">
    document.addEventListener("adobe_dc_view_sdk.ready", function () {
        var adobeDCView = new AdobeDC.View({
            clientId: "<<YOUR_CLIENT_ID>>",
            divId: "adobe-dc-view"
        });
        adobeDCView.previewFile({
            content: { location: { url: "<<PDF Location>>" } },
            metaData: { fileName: "<<PDF File Name>>" }
        }, {
            embedMode: "IN_LINE",
            showDownloadPDF: false,
            showPrintPDF: false
        });
    });
</script>

Check out the official docs and the code repository for more samples.

Write Google Scripts using Modern JavaScript ES6

Google Apps Script has received a significant upgrade ever since it was first released to the public more than a decade ago. Apps Script now uses the V8 JavaScript Engine - it is the same runtime that is used inside the Google Chrome browser and the popular Node.js environment.

This essentially means:

  • Developers can write code using modern JavaScript syntax like Arrow Functions, Classes, Array Destructuring, Template Literals and more.
  • The V8 Engine is fast, powerful and continuously improving, It will likely improve the performance and memory utilization of your Google Scripts.
  • Developers can use new JavaScript ES6 features like Symbols, Iterators, Generators, Promises, Maps, Sets and Proxies that weren’t available in the previous version of Google Apps Script.

Chrome V8 JavaScript Engine

Any new projects that you create inside the Google Apps Script editor automatically use the new V8 runtime.

V8 JavaScript Runtime in Google Apps Script

If you would like to upgrade any old project to use V8, go to the Run menu and choose “Enable new Apps Script runtime powered by V8.” If you do not see this option yet, add a new runtimeVersion field in your project’s manifest file with the value of V8. You can set the value to DEPRECATED_ES5 to switch to the old version that uses the Mozilla’s Rhino JavaScript engine.

Tip: Type script.new in your browser to quickly create a new Google Apps Script project in your browser. (Source)

ES6 Modules - The Missing Part

ES6 introduced the concept of modules in JavaScript which allows developers to write reusable code that is also easier to refactor and maintain. You can break your program into separate files (modules) and then import them into other modules using import-export statements.

The new Google Apps Script environment doesn’t support ES6 modules.

The other big change is that functions become available based on the sequence of files in the script editor. Let me explain.

Say your Apps Script project has a lot of files and you have created two functions of the same name but they are located in different files. Apps Script won’t complain but when you can call this function, the one that is defined in the bottom-most file of the project will be invoked.

Google Apps Script Modules

If you prefer the ease of working with ES6 Modules, the Apps Script Starter kit can help. You can write code locally inside Visual Studio Code, bundle the modules into a single file with Webpack and then push the bundle to the cloud automatically with Clasp.

The starter kit has also been updated to use the new V8 Runtime. Watch this YouTube video to learn how to develop with Google Scripts using the Starter Kit.

Performance - V8 vs Vanilla JavaScript

Eric Koleda writes - “The performance story is mixed. Vanilla JavaScript code (looping, math) runs faster, but calls to G Suite services (SpreadsheetApp, etc) run slightly slower. While not ideal, we’ve always recommended that performance-intensive apps are usually a better fit for other platforms. In general Apps Script is trying to optimize for ease-of-use, not throughput. If performance is critical to your use case then you may want to investigate Google Cloud Functions, etc.”

Learn Modern JavaScript ES6

Coming back to Javascript, I do have a few recommendations that will help improve your understanding of ECMAScript 6.

  • Understanding ES6 - This online book covers all the new features that have been added to the JavaScript language since ES6.
  • ES6 Udacity - A detailed video course that covers all aspects of ES6, complete with quizzes and doesn’t cost a penny.
  • Exploring ES6 - Deep dive into the core ES6 features with examples.
  • ES6+ Introduction - An interactive screencast tutorial series that provides a walk-through of the most important ES6+ features.
  • Mozilla Docs - MDN is the best reference site for JavaScript, including ES6.
  • If you prefer premium courses, check out the ones by Maximilian Schwarzmüller, Wes Bos and Stephen Grider.

Also see: The Best Online Teachers for Web Development

The Easiest Way to Extract Email Addresses from your Gmail Account

Introducing Gmail Address Extractor, a web app that parses email messages in your Gmail mailbox, finds all the email addresses in them and stores the list in a Google Sheet. You can export the sheet as a CSV file and import into Google Contacts, Outlook address book, MailChimp, or any other mailing list software.

The online extractor app can find email addresses that are contained in the message body, the email signature, the subject line and the FROM, TO, CC, BCC & Reply-To fields of the email message. The app cannot, however, extract emails from file attachments.

gmail address extractor

Getting started is easy and requires no software installation. Just go to emailextract.pro and sign-in with your Gmail or Google Inbox account. You can also sign with your G Suite (Google Apps) account.

The app can pull email addresses from any label in your Gmail account or you can specify advanced search criteria and email addresses would only be extracted from the matching email threads. For instance, you could set the advanced search rule as from:paypal newer_than:7d and only PayPal emails received in the last week will be processed.

You can extract email addresses from messages that come via form submissions, email addresses of customers that are found inside the PayPal and Stripe receipts, email addresses of your newsletter subscribers and so on.

If you would like to extract email addresses from all Gmail folders in one go, choose “Anywhere” in the label drop-down.

After you’ve specified the search criteria, you need to select the email fields for extraction and the destination Google Sheet where the email addresses would be saved. The sheet will automatically remove any duplicate email addresses that it may encounter during the extraction.

extract-email-fields.png

Once the email addresses are in the Google Sheet, you can use filters inside sheets to exclude addresses from certain domains.

Email Extractor runs locally in your browser and not a byte of your email data is shared or uploaded anywhere. It does require a one-time authorization to various Google services since the app needs to read Gmail messages and save the list in a Google sheet in your Google Drive. You can read the app’s privacy policy to learn more.

The extractor app is completely free for lite users if you need to process less than 300 email messages. For large mailboxes, the pricing is $29 USD and you can process an unlimited number of email threads with the paid edition.

The app is also available as a Gmail addon in the Google Store. It works with Gmail only but if you can import your Yahoo or Outlook email message into Gmail, the addon can process them as well.

Also see: Save Gmail Messages in Google Drive

How to Create Forms that Allow File Uploads to Google Drive

With File Upload Forms (demo) for Google Sheets, you can receive large files of any size from anyone directly in your Google Drive. You can automatically send email confirmations to notify the form submitter. The forms can be created inside Google Sheets using the built-in drag-n-drop form builder and the forms can have CAPTCHAs, e-signature, multiple file uploads, password protection and more.

Unlike the file uploads feature in Google Forms, the form respondents do not require a Google account to uploads files through your form.

file-upload-forms.png

Receive Files from Anyone in Google Drive

A school teacher may want to build forms for students to upload assignments and the files are automatically saved to her Google Drive but in separate student folders. The HR team may want an online form where job applicants can upload their resumes in PDF or Word format. Business can build forms with eSignatures that respondents can sign on their desktops and mobile phones.

Google Forms do allow file uploads but the respondents need to be signed into their Google accounts before they can upload files. File Upload Forms impose no such limitations - anyone can upload files to your Google Drive. They may not have a Google account and they’ll still be able to use your Drive uploader forms.

File Upload Forms - Getting Started

Here’s a step-by-step guide on how you can build your own File Upload Forms in a few minutes. Or watch the YouTube video to get started.

Step 1. Copy the Google Sheet for File Upload Forms

The File Upload Form is written in Google Scripts and the code needs to be attached to your Google Sheet for it to work.

To get started, go to forms.studio/copy and click the “Copy” button to create a copy of the Google Sheet template to your Google Drive. This sheet includes the form builder, email designer and it will also be storing your form responses just like Google Forms responses are stored inside Google Sheets.

Step 2. Deploy the Uploader form as Web App

Inside your Google Spreadsheet, go to Tools -> Script Editor to open the Google Script editor. Don’t worry, you don’t have to write a single line of code, you only have to deploy this script as a web app so anyone can access your form via a simple web URL.

Go to Publish -> Deploy as Web App, choose Me under Execute the app as, choose Anyone, even Anonymous under Who has access to the web app and click the Deploy button.

google-form-deploy-web-app.png

You may have to authorize the script once since it has to do all the operations - like sending emails, uploading files - on your behalf.

Also, if you get the “App Not Verified” screen, just click on the Advanced link and choose “Go to File Upload Forms” to continue. While this step is optional, you can submit your app to Google for verification and they’ll drop the ‘not verified’ warning.

Step 3. Design the File Uploader Form

Switch to the Google Sheet, expand the File Upload Forms menu (near Help) and choose Edit Form. It will launch a simple but powerful drag-n-drop form builder right inside your Google Sheets.

File Upload Form - Design Form

Your forms can have multiple file upload fields, eSignatures, date, time picker, and all the standard form fields. Use the Rich HTML field to embed YouTube videos, Google Maps, SoundCloud MP3s, Google Slide presentations, Images or any HTML content.

You can also add data validation rules to various form fields using simple regular expressions. The file upload fields can be configured to accept single or multiple files. You can also limit uploads to specific types like images, videos, zip, and more.

Step 4. Customize the Form’s Settings

After your form is designed, go back to the File Upload Forms menu and choose Configure Form. It will open a sidebar in your Google Sheet where you can customize the form and Google Drive settings.

The confirmation message can include place-markers - form fields enclosed inside double curly braces like {{name}} - for personalized messages. You can also place a URL inside the Redirect option and respondents would be redirected to that website after submitting the form.

file-upload-form-settings.png

In the advanced settings section, click the Select Folder button to choose the parent folder in your Google Drive where all the files would be saved. You can also specify a sub-folder path for storing files and this path can be dynamic - \\{{Country}}\\{{City}} - based on form answers.

You can also change color schemes, track visits to your form with Google Analytics and protect your forms with Google CAPTCHA and passwords.

Step 5. Send Confirmation Emails

With File Upload Forms, you can send email notifications when people submit the form. The emails can be personalized by including the form fields in the email subject and message body.

Use the special {{All Answers}} place marker in the message body to include a summary of the form response in a neat table. The TO, CC or BCC field can include the {{question title}} where you ask for the email address and it will be replaced with the actual value in the form response.

drive-uploader-forms-email.png

That’s it.

Your form is now ready for the world. Expand the Share section and you’ll be given the URL to directly access the form and also the embed code for adding that form to any website.

You can use Gmail Mail Merge to send the form link to all your contacts in a personalized email. The  Document Studio add-on can help you create beautiful, pixel-perfect PDFs from the form response and they are auto-saved in Google Drive.

Things to Know - File Upload Forms

  • To stop accepting new responses, go to Publish -> Deploy as web app menu and click the Disable link.
  • If you wish to restrict the forms to users inside your GSuite organization, choose your domain under Who has access to the app in place of Anonymous.

File Upload Forms - Compare Plans

File Upload Forms for Google Drive require a license for extended use. The basic version is free forever.

FreePremium (Upgrade)
Form ResponsesUnlimitedUnlimited
Drag & Drop Form Builder No Yes
Customized Confirmations No Yes
Email Notifications No Yes
Redirect on Form Submit No Yes
Multiple Color Schemes No Yes
Google CAPTCHA No Yes
Save Files in Custom Folders No Yes
RegEx Input Validation No Yes
Upload Multiple Files No Yes
E-Signature Fields No Yes
Capture Browser’s User Agent No Yes
Localize Forms in different languages No Yes
Technical Support No Email
PricingFreeUpgrade to Premium

Count the Number of Words and Characters in a Google Document

If you were to count the number of words and characters in a Google Document, open the document, go to the Tools menu and choose Word Count. That’s a good option for counting words in a single document manually but what if you have a folder of files in Google Drive, say student assignments, and wish to know the words or characters per document.

That’s where Google Apps Script can help.

Go to Tools > Script Editor and paste the code to programmatically get the word count of any document in Google Document. You can either provide the document ID to the function or it will use the currently opened document.

function getWordCount(fileId) {
  const SEPARATOR = ' ';
  const document = fileId
    ? DocumentApp.openById(fileId)
    : DocumentApp.getActiveDocument();
  const text = document.getBody().getText();
  const words = text.replace(/\s+/g, SEPARATOR).split(SEPARATOR);
  const characters = words.join('');
  Logger.log("Word Count: " + words.length);
  Logger.log("Character Length: " + characters.length);
}

A more advanced version of the function uses regular expressions and it can work with Chinese, Japanese and Korean scripts - Credit.

function getWordCountCJK(data) {
  var pattern = /[a-zA-Z0-9_\u0392-\u03c9]+|[\u4E00-\u9FFF\u3400-\u4dbf\uf900-\ufaff\u3040-\u309f\uac00-\ud7af]+/g;
  var m = data.match(pattern);
  var count = 0;
  if( m === null ) return count;
  for (var i = 0; i < m.length; i++) {
    if (m[i].charCodeAt(0) >= 0x4E00) {
      count += m[i].length;
    } else {
      count += 1;
    }
  }
  return count;
}


function getWordCount(fileId) {
  const SEPARATOR = ' ';
  const document = fileId
    ? DocumentApp.openById(fileId)
    : DocumentApp.getActiveDocument();
  const text = document.getBody().getText();
  const count = getWordCountCJK(text);
  Logger.log("Word Count: " + count);
}