Should we use permanent links when citing online sources?

Question

Online resources, including Wikipedia articles, arXiv articles and other web pages are often cited and used in answers and questions here.

Questions and answers on this site are expected to help future visitors. But, many of these online webpages are subject to change, modification or even removal. For example, Wikipedia articles may change significantly and do not contain (exactly) the cited material any more and arXiv articles may also change duo to uploading a newer version. The problem is much more severe with other web pages, where their address may change or the whole page may be removed.

For Wikipedia and arXiv articles, a permanent link to each version is provided by the site itself.

For (many) other pages, the solution is to use a web archiving service, like WebCite to prevent link rot.

Shouldn't we use permanent links/archived versions of cited online resources in questions/answers? I know that questions and answers are expected to be self-contained as much as possible, but citing a more comprehensive online source is usually unavoidable.

Also, it can be done automatically by a bot.

(For a discussion on the same issue on Wikipedia, see here.)

Answer 1

I think this is not a particularly big problem given the nature of most common links in this site.

Regarding your two examples, Wikipedia and arXiv updates are natural and linking to an old version is bad practice. If the Wikipedia edits are radical enough that the original point is lost then there is always the edit history both on their side and ours that it's possible to reconstruct what the original link was. ArXiv edits are very rarely substantive enough to remove crucial information, and if they are it is a strong indication that that content was unreliable (and, again, there is the edit history to fall back on).

For other links, it's more of a case-by-case thing. The Wayback Machine now enables visitors to archive pages (unless forbidden by the site's robots.txt file), and it is generally good practice to make sure that the Wayback Machine has a suitable copy archived before linking to a webpage on this site, and particularly if the link looks susceptible to link rot. However, the link should always point to the current live version. If the link rots in the future, or the target page changes, then the link can be re-directed to the archived version if and when than happens.

Another class of links which is more fragile (if relatively less common on this site) is YouTube videos, which can disappear for a variety of reasons and leave essentially no traces behind them (stable image). If you're linking to a video, make sure that the video's title is preserved either in your post or in an archived version of the page. Archival services, in general, won't archive the video, but they make it possible to understand what the link was and, if there is an equivalent video available, to redirect the link to the new location.

So, in general, here are some guidelines for when and how to take active steps to prevent link rot:

• Always use DOI links when linking to online articles. A Digital Object Identifier is a permanent link and will always follow the document, while journal website urls can change and even be inaccessible to users in other locations. Find the DOI marker on the page, e.g. doi:10.1038/nature12422, and link to http://dx.doi.org/10.1038/nature12422. For books that don't have DOI's, a good alternative is an Open Library url.
• While we're at this, when linking to the arXiv, link to the abstract page (arxiv.org/abs/...) and not directly to the pdf. This is a courtesy to users on lower bandwidths and mobile browsers (which can automatically redirect to separate pdf viewer apps), who can then evaluate, from the abstract page, whether they want to actually download the paper.
• If the paper has been published (which will be noted on the arXiv abstract), include a link to the journal version, even if it's paywalled, if only as a courtesy to the referees' hard work. In some fields of physics it is standard practice to upload the last, published version to the arXiv, but this is not true in all fields. Moreover, journals collect download and usage statistics and make them available to the authors, which the arXiv does not do. In any case, linking to the official, journal version (as well as the arXiv!) helps the author by improving the search engine ranking of their paper, and it probably helps us in that regard as well.
• If citing a pdf of a paper or similar that is not on the arXiv or a similar repository (e.g. a university repository), use a web citation service to protect it, ideally the Wayback Machine, or at least make a local copy.
• When citing Wikipedia or the arXiv, always cite the most recent version. Revisions will most likely be improvements, and if it is unclear what the reference was for there is enough edit history to reconstruct that.
• When citing external websites, always cite the page's current position, and make sure that there is an appropriate version of the page in the Wayback Machine.
• When linking to YouTube or similar services, make sure that the title and other information of the content are preserved in your post, to make the linked content easier to trace if it vanishes.

Demanding permanent links is rather too harsh and will slow down the site, particularly with new users. If someone posts a particularly fragile link, change it to a more permanent one (using WebCite, the Wayback Machine, or similar if appropriate) and refer them to this question. More experienced users should be proactive in editing such links when they look fragile.

Answer 2

We certainly endorse that users amend their posts with relevant links, as long as the post is self-contained, and not just a link farm.

Permalinks should not be demanded, but it should definitely be encouraged.

Phys.SE is still less than 3 years old, but link rot may become a problem in the future.

Answer 3

Permalinks to frozen versions of references should be actively discouraged.

When a WP article changes, the change is probably an improvement (or at least this is true to the extent that WP is a system that manages to work as intended). When a paper on arxiv changes, the change is almost certainly an improvement. If we point people to out of date versions, the most common effect will be none whatsoever, while the second most common effect will be to make their lives worse by, e.g., letting them take an equation out of a paper without realizing that there was a wrong factor of 2 -- the factor of 2 being irrelevant to the SE discussion, but potentially a source of pain and suffering for the unwitting user of the permalink.

Wikipedia articles may change significantly and do not contain (exactly) the cited material any more

But everybody understands this and won't be particularly surprised. There might be more of an issue if we were using WP as a definitive source of information that would settle any argument for once and for all -- but of course we're not doing that.

We're not referencing historical documents or holy texts. SE answers are supposed to be self-contained and valid on their own merits.

An option that deserves serious consideration is that if you have a PDF of a paper, you can put it on the web for others to use. This could be legal or illegal (depending on your country's laws on fair use, etc.), moral or immoral (depending on your opinion of the morality of the current copyright regime). I sometimes put papers, especially historical ones, on scribd.com. When I link to them from SE, I try to remember to warn people of the potential copyright issue if they click through on the link.