NgsRelate - Revision history

Thorfinn at 13:59, 5 October 2018

2018-10-05T13:59:09Z

← Older revision		Revision as of 15:59, 5 October 2018
Line 1:		Line 1:
			= NEW VERSION =
	For the NEW version of ngsRelate that coestimates relatedness and inbreeding go to this link https://github.com/ANGSD/NgsRelate		For the NEW version of ngsRelate that coestimates relatedness and inbreeding go to this link https://github.com/ANGSD/NgsRelate


			= OLD VERSION =
	For the old version please use this link: http://www.popgen.dk/software/index.php?title=NgsRelate&oldid=694		For the old version please use this link: http://www.popgen.dk/software/index.php?title=NgsRelate&oldid=694

Thorfinn: Replaced content with "For the NEW version of ngsRelate that coestimates relatedness and inbreeding go to this link https://github.com/ANGSD/NgsRelate For the old version please use this link: h..."

2018-10-05T13:58:33Z

Replaced content with "For the NEW version of ngsRelate that coestimates relatedness and inbreeding go to this link https://github.com/ANGSD/NgsRelate For the old version please use this link: h..."

Show changes

Thorfinn at 12:48, 3 October 2018

2018-10-03T12:48:00Z

← Older revision		Revision as of 14:48, 3 October 2018
Line 125:		Line 125:

	= Citing and references =		= Citing and references =
			Method (v1) is published here: http://bioinformatics.oxfordjournals.org/content/early/2015/08/29/bioinformatics.btv509.abstract
	= Changelog =		= Changelog =
	Important recent changes:		Important recent changes:

Thorfinn: /* Run example 3: using frequencies from 1000genomes vcf files */

2018-10-03T12:46:54Z

Run example 3: using frequencies from 1000genomes vcf files

← Older revision		Revision as of 14:46, 3 October 2018
Line 43:		Line 43:

	'''NEW''': Note that if you want you also input a file with the IDs of the individuals (on ID per line) in the same order as in the file 'filelist' used to make the genotype likelihoods. If you do the output will also contain these IDs and not just the numbers of the samples (one can actually just use that exact file, however the IDs then tend to be a bit long). This can be done with the optional flag -z followed by the filename.		'''NEW''': Note that if you want you also input a file with the IDs of the individuals (on ID per line) in the same order as in the file 'filelist' used to make the genotype likelihoods. If you do the output will also contain these IDs and not just the numbers of the samples (one can actually just use that exact file, however the IDs then tend to be a bit long). This can be done with the optional flag -z followed by the filename.

	~~== Run example 3: using frequencies from 1000genomes vcf files==~~
	~~We want to run ngsRelate using population frequencies from europe. We will extract the frequencies from the 1000genomes project vcf.~~

	~~<pre>~~
	~~#Assuming that we have perchr called: ALL.chr*.phase3_shapeit2_mvncall_integrated_v5.20130502.genotypes.vcf.gz~~
	~~#We dump output in EUR_AF/*.frq~~
	~~#We only use diallelic sites, we extract CHROM,POS,REF,ALT,EUR_AF tags from the vcf~~
	~~#We then pulled out the unique sites.~~
	~~for f in `seq 1 22`~~
	do
	~~IF=/storage/data_shared/callsets/1000genomes/phase3/vcf/ALL.chr${f}.phase3_shapeit2_mvncall_integrated_v5.20130502.genotypes.vcf.gz~~
	~~echo "bcftools view -m2 -M2 -v snps ${IF} \| bcftools query -f '%CHROM\t%POS\t%REF\t%ALT\t%EUR_AF\n' \|awk '{if(\$5>0) print \$0 }'\|sort -S 50% -u -k1,2 >EUR_AF/${f}.frq"~~
	~~done\|parallel~~

	~~##We merge into one file~~
	~~cat EUR_AF/1.frq >EUR_AF/ALL.frq~~
	~~for i in `seq 2 22`~~
	do
	~~cat EUR_AF/${i}.frq >>EUR_AF/ALL.frq~~
	~~done~~
	~~gzip EUR_AF/ALL.frq~~

	~~#we extract the first 4 columns, which is the sites input for angsd~~
	~~gunzip -c EUR_AF/ALL.frq.gz \|cut -f1-4 \|gzip -c >EUR_AF/sites.txt.gz~~

	~~./angsd/angsd sites index EUR_AF/sites.txt.gz~~
	~~./angsd/angsd -b list -gl 1 -domajorminor 3 -C 50 -ref /storage/data_shared/reference_genomes/hs37d5/hs37d5.fa -doglf 3 -minmapq 30 -minq 20 -sites EUR_AF/sites.txt.gz~~

	~~#Then we extract and match the freqs from the reference population with the sites where we had data. The parser expects a header, so make a dummy file~~

	~~echo "header" \|gzip -c >new~~
	~~cat EUR_AF/ALL.frq.gz >>new~~
	~~ngsRelate extract_freq new angsdput.glf.pos.gz >myfreq~~


	~~</pre>~~

	=Output format=		=Output format=

Thorfinn: /* Run examples */

2018-10-03T12:46:26Z

Run examples

← Older revision		Revision as of 14:46, 3 October 2018
Line 23:		Line 23:
	</pre>		</pre>

	The output should be a file called res that contains relatedness estimates for all pairs between 6 individuals~~. A copy of this file can be found here http://www.popgen.dk/ida/NgsRelateExampleData/web/output/newres~~.		The output should be a file called res that contains relatedness estimates for all pairs between 6 individuals.

	== Run example 0: using only VCF/BCF files==		== Run example 0: using only VCF/BCF files==

Thorfinn: /* Run examples */

2018-10-03T12:45:56Z

Run examples

← Older revision		Revision as of 14:45, 3 October 2018
Line 18:		Line 18:

	=Run examples=		=Run examples=
	Below are ~~two~~ examples of how NgsRelate can be used to ~~estimate~~ relatedness from NGS data. Note that to be able to run all steps of the examples you need to have the programs ANGSD and PLINK installed and you also need to download large data files from both HapMap3 and 1000 Genomes webpages. Furthermore, the examples take several hours to run all in all. They are therefore just meant as illustrations of how NgsRelate can be run. '''If you want to quickly try out NgsRelate, e.g. to check if your installation works, you can download the final input data for NgsRelate used in the very last command in run example 2 here: http://www.popgen.dk/ida/NgsRelateExampleData/web/input/. Using that data you can try out NgsRelate by running that last command, i.e.'''		Below are examples of how NgsRelate can be used to coestimate relatedness and inbreeding from NGS data.

	<pre>		<pre>
	./ngsrelate -g angsdput.glf.gz -n 6 -f freq > newres		./ngsrelate -g angsdput.glf.gz -n 6 -f freq > newres

Thorfinn: /* Run example 0: using only BCF files */

2018-10-03T12:44:24Z

Run example 0: using only BCF files

← Older revision		Revision as of 14:44, 3 October 2018
Line 26:		Line 26:
	The output should be a file called res that contains relatedness estimates for all pairs between 6 individuals. A copy of this file can be found here http://www.popgen.dk/ida/NgsRelateExampleData/web/output/newres.		The output should be a file called res that contains relatedness estimates for all pairs between 6 individuals. A copy of this file can be found here http://www.popgen.dk/ida/NgsRelateExampleData/web/output/newres.

	== Run example 0: using only BCF files==		== Run example 0: using only VCF/BCF files==

	== Run example 1: using only NGS data==		== Run example 1: using only NGS data==

Thorfinn at 12:43, 3 October 2018

2018-10-03T12:43:56Z

← Older revision		Revision as of 14:43, 3 October 2018
Line 44:		Line 44:

	'''NEW''': Note that if you want you also input a file with the IDs of the individuals (on ID per line) in the same order as in the file 'filelist' used to make the genotype likelihoods. If you do the output will also contain these IDs and not just the numbers of the samples (one can actually just use that exact file, however the IDs then tend to be a bit long). This can be done with the optional flag -z followed by the filename.		'''NEW''': Note that if you want you also input a file with the IDs of the individuals (on ID per line) in the same order as in the file 'filelist' used to make the genotype likelihoods. If you do the output will also contain these IDs and not just the numbers of the samples (one can actually just use that exact file, however the IDs then tend to be a bit long). This can be done with the optional flag -z followed by the filename.

	~~== Run example 2: using NGS data with population frequencies estimated from genetic data from PLINK files ==~~
	~~In this example we show how you can estimate relatedness between a number of individuals which you have NGS data from (in bam files) using genetic data from PLINK files for frequency estimation.~~
	Assume the individuals we want to estimate relatedness from are from the population called LWK and assume we have files with genetic data from individuals from LWK as well as other populations in binary PLINK format (e.g. hapmap3_r2_b36_fwd.consensus.qc.polyHg19.*) and a file, LWK.fam, with the IDs of the LWK individuals in this dataset. Then using PLINK we can produce allele frequency information in a format that NgsRelate can use as follows:
	~~<pre>~~
	~~### extract individuals from LWK from huge binary plink file~~
	~~plink --bfile hapmap3_r2_b36_fwd.consensus.qc.polyHg19 --keep LWK.fam --make-bed --out hapmap3Hg19LWK --noweb~~

	~~### calculate frequencies for this population~~
	~~plink --bfile hapmap3Hg19LWK --freq --noweb --out LWKsub~~
	~~</pre>~~
	~~Afterwards we can use ANGSD to calculate genotype likelihoods for the sites for which we have frequency info for as follows:~~
	~~<pre>~~
	~~### extract the chr,pos,major,minor information about the sites we have frequency info from into a file~~
	~~### (so we can extract data from these sites from the NGS data files)~~
	~~cut -f1,4-6 hapmap3Hg19LWK.bim >forAngsd.txt~~

	~~### index this file for angsd~~
	~~./angsd sites index forAngsd.txt~~

	~~### calculate genotype likelihoods for the six individuals for the sites we have frequency info on based on the bam files~~
	~~### (assuming the paths to the bam files are listed in the file 'list'):~~
	~~./angsd -gl 1 -doglf 3 -sites forAngsd.txt -b list -domajorminor 3 -P 2 -minMapQ 30 -minQ 20~~
	~~### this generates the output files angsdput.glf.gz and a angsdput.glf.pos.gz.~~
	~~</pre>~~

	~~Finally we can use NgsRelate to estimate relatedness for the six individuals from which we have NGS data in bam files:~~
	~~<pre>~~
	~~### extract the frequencies and sync it to the angsd output~~
	~~./ngsrelate extract_freq_bim angsdput.glf.pos.gz hapmap3Hg19LWK.bim LWKsub.frq >freq~~

	~~### run ngsrelate~~
	~~./ngsrelate -g angsdput.glf.gz -n 6 -f freq >newres~~
	~~</pre>~~
	~~The final relatedness estimates will then be available in the file called "newres" which can be found here: http://www.popgen.dk/ida/NgsRelateExampleData/web/output/.~~

	== Run example 3: using frequencies from 1000genomes vcf files==		== Run example 3: using frequencies from 1000genomes vcf files==

Thorfinn at 12:42, 3 October 2018

2018-10-03T12:42:11Z

← Older revision		Revision as of 14:42, 3 October 2018
Line 1:		Line 1:
			This pages refers to the new v2 of ngsRelate which coestimates relatedness and inbreeding. For the old version please use this link: http://www.popgen.dk/software/index.php?title=NgsRelate&oldid=694

	=Brief description=		=Brief description=
	This page contains information about the program called NgsRelate, which can be used to infer relatedness coefficients for pairs of individuals from low coverage Next Generation Sequencing (NGS) data by using genotype likelihoods instead of called genotypes. To be able to infer the relatedness you will need to know the population frequencies and have genotype likelihoods. This can be obtained e.g. using the program ANGSD as shown in the examples below. For more information about ANGSD see here: http://popgen.dk/angsd/index.php/Quick_Start.		This page contains information about the program called NgsRelate, which can be used to infer relatedness coefficients for pairs of individuals from low coverage Next Generation Sequencing (NGS) data by using genotype likelihoods instead of called genotypes. To be able to infer the relatedness you will need to know the population frequencies and have genotype likelihoods. This can be obtained e.g. using the program ANGSD as shown in the examples below. For more information about ANGSD see here: http://popgen.dk/angsd/index.php/Quick_Start.

Thorfinn: /* Run example 0: using only NGS data */

2018-07-30T09:39:58Z

Run example 0: using only NGS data

← Older revision		Revision as of 11:39, 30 July 2018
Line 24:		Line 24:
	The output should be a file called res that contains relatedness estimates for all pairs between 6 individuals. A copy of this file can be found here http://www.popgen.dk/ida/NgsRelateExampleData/web/output/newres.		The output should be a file called res that contains relatedness estimates for all pairs between 6 individuals. A copy of this file can be found here http://www.popgen.dk/ida/NgsRelateExampleData/web/output/newres.

	== Run example 0: using only ~~NGS data~~==		== Run example 0: using only BCF files==

	== Run example 1: using only NGS data==		== Run example 1: using only NGS data==